.. index:: pair: struct; dnnl::primitive_attr .. _doxid-structdnnl_1_1primitive__attr: struct dnnl::primitive_attr =========================== .. toctree:: :hidden: Overview ~~~~~~~~ Primitive attributes. :ref:`More...` .. ref-code-block:: cpp :class: doxyrest-overview-code-block #include struct primitive_attr: public :ref:`dnnl::handle` { // construction :ref:`primitive_attr`(); :ref:`primitive_attr`(:ref:`dnnl_primitive_attr_t` attr); // methods :ref:`fpmath_mode` :ref:`get_fpmath_mode`() const; void :ref:`set_fpmath_mode`(:ref:`fpmath_mode` mode); :ref:`scratchpad_mode` :ref:`get_scratchpad_mode`() const; void :ref:`set_scratchpad_mode`(:ref:`scratchpad_mode` mode); void :ref:`get_output_scales`(int& mask, std::vector& scales) const; void :ref:`set_output_scales`(int mask, const std::vector& scales); void :ref:`get_scales`(int arg, int& mask, std::vector& scales) const; void :ref:`set_scales`(int arg, int mask, const std::vector& scales); void :ref:`get_zero_points`(int arg, int& mask, std::vector& zero_points) const; void :ref:`set_zero_points`(int arg, int mask, const std::vector& zero_points); const :ref:`post_ops` :ref:`get_post_ops`() const; void :ref:`set_post_ops`(const :ref:`post_ops` ops); void :ref:`set_rnn_data_qparams`(float scale, float shift); void :ref:`get_rnn_data_qparams`(float& scale, float& shift); void :ref:`set_rnn_weights_qparams`(int mask, const std::vector& scales); void :ref:`get_rnn_weights_qparams`(int& mask, std::vector& scales); void :ref:`set_rnn_weights_projection_qparams`( int mask, const std::vector& scales ); void :ref:`get_rnn_weights_projection_qparams`(int& mask, std::vector& scales); }; Inherited Members ----------------- .. ref-code-block:: cpp :class: doxyrest-overview-inherited-code-block public: // methods :ref:`handle`& :ref:`operator =` (const :ref:`handle`&); :ref:`handle`& :ref:`operator =` (:ref:`handle`&&); void :ref:`reset`(T t, bool weak = false); T :ref:`get`(bool allow_empty = false) const; :ref:`operator T` () const; :ref:`operator bool` () const; bool :ref:`operator ==` (const :ref:`handle`& other) const; bool :ref:`operator !=` (const :ref:`handle`& other) const; .. _details-structdnnl_1_1primitive__attr: Detailed Documentation ~~~~~~~~~~~~~~~~~~~~~~ Primitive attributes. .. rubric:: See also: :ref:`Primitive Attributes ` Construction ------------ .. index:: pair: function; primitive_attr .. _doxid-structdnnl_1_1primitive__attr_1acfbfd85b7ca82bf97e2b07c2427427de: .. ref-code-block:: cpp :class: doxyrest-title-code-block primitive_attr() Constructs default (empty) primitive attributes. .. index:: pair: function; primitive_attr .. _doxid-structdnnl_1_1primitive__attr_1aafb54e73f3abe59555f1cfe62407280e: .. ref-code-block:: cpp :class: doxyrest-title-code-block primitive_attr(:ref:`dnnl_primitive_attr_t` attr) Creates primitive attributes from a C API :ref:`dnnl_primitive_attr_t ` handle. The resulting handle is not weak and the C handle will be destroyed during the destruction of the C++ object. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - attr - The C API primitive attributes. Methods ------- .. index:: pair: function; get_fpmath_mode .. _doxid-structdnnl_1_1primitive__attr_1af335f8e1e74b69c5b2d4da0ecf8a23d5: .. ref-code-block:: cpp :class: doxyrest-title-code-block :ref:`fpmath_mode` get_fpmath_mode() const Returns the fpmath mode. .. index:: pair: function; set_fpmath_mode .. _doxid-structdnnl_1_1primitive__attr_1a31f2e897b523ee6265ef9b718799994b: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_fpmath_mode(:ref:`fpmath_mode` mode) Sets fpmath mode. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mode - Specified fpmath mode. .. index:: pair: function; get_scratchpad_mode .. _doxid-structdnnl_1_1primitive__attr_1af4131b946ec3af3bc2974b603d30029b: .. ref-code-block:: cpp :class: doxyrest-title-code-block :ref:`scratchpad_mode` get_scratchpad_mode() const Returns the scratchpad mode. .. index:: pair: function; set_scratchpad_mode .. _doxid-structdnnl_1_1primitive__attr_1a91a597649afa13b7d2416b708d0620d2: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_scratchpad_mode(:ref:`scratchpad_mode` mode) Sets scratchpad mode. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mode - Specified scratchpad mode. .. index:: pair: function; get_output_scales .. _doxid-structdnnl_1_1primitive__attr_1aadcfc1f7a787a0b75d6b0daab8da14af: .. ref-code-block:: cpp :class: doxyrest-title-code-block void get_output_scales(int& mask, std::vector& scales) const Returns output scaling factors correspondence mask and values. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mask - Scaling factors correspondence mask that defines the correspondence between the output tensor dimensions and the ``scales`` vector. The set i-th bit indicates that a dedicated output scaling factor is used for each index along that dimension. The mask value of 0 implies a common output scaling factor for the whole output tensor. * - scales - Vector of output scaling factors. .. index:: pair: function; set_output_scales .. _doxid-structdnnl_1_1primitive__attr_1a4b81acc8e48886313154f75c1708ae02: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_output_scales(int mask, const std::vector& scales) Sets output scaling factors correspondence mask and values. Example usage: .. ref-code-block:: cpp int mb = 32, oc = 32, oh = 14, ow = 14; // convolution output params // unique output scales per output channel vector scales = { ... }; int oc_dim = 1; // mb_dim = 0, channel_dim = 1, height_dim = 2, ... // construct a convolution descriptor dnnl::convolution::desc conv_d; :ref:`dnnl::primitive_attr ` attr; attr.:ref:`set_output_scales `(attr, oc, 1 << oc_dim, scales); :ref:`dnnl::primitive_desc ` conv_pd(conv_d, attr, engine); .. note:: The order of dimensions does not depend on how elements are laid out in memory. For example: * for a 2D CNN activations tensor the order is always (n, c) * for a 4D CNN activations tensor the order is always (n, c, h, w) * for a 5D CNN weights tensor the order is always (g, oc, ic, kh, kw) .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mask - Defines the correspondence between the output tensor dimensions and the ``scales`` vector. The set i-th bit indicates that a dedicated scaling factor is used for each index along that dimension. Set the mask to 0 to use a common output scaling factor for the whole output tensor. * - scales - Constant vector of output scaling factors. If the scaling factors are known at the time of this call, the following equality must hold: :math:`scales.size() = \prod\limits_{d \in mask} output.dims[d].` Violations can only be detected when the attributes are used to create a primitive descriptor. If the scaling factors are not known at the time of the call, this vector must contain a single :ref:`DNNL_RUNTIME_F32_VAL ` value and the output scaling factors must be passed at execution time as an argument with index :ref:`DNNL_ARG_ATTR_OUTPUT_SCALES `. .. index:: pair: function; get_scales .. _doxid-structdnnl_1_1primitive__attr_1a9fc7eb58cc958963a2d26743cfd5c763: .. ref-code-block:: cpp :class: doxyrest-title-code-block void get_scales(int arg, int& mask, std::vector& scales) const Returns scaling factors correspondence mask and values for a given memory argument. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - arg - Parameter argument index as passed to the :ref:`primitive::execute() ` call. * - mask - Scaling factors correspondence mask that defines the correspondence between the output tensor dimensions and the ``scales`` vector. The set i-th bit indicates that a dedicated scaling factor is used for each index along that dimension. Set the mask to 0 to use a common scaling factor for the whole output tensor. * - scales - Output vector of scaling factors. .. index:: pair: function; set_scales .. _doxid-structdnnl_1_1primitive__attr_1a9bf717bd25b6fddd89055da8178eab75: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_scales(int arg, int mask, const std::vector& scales) Sets scaling factors for primitive operations for a given memory argument. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - arg - Parameter argument index as passed to the :ref:`primitive::execute() ` call. * - mask - Scaling factors correspondence mask that defines the correspondence between the tensor dimensions and the ``scales`` vector. The set i-th bit indicates that a dedicated scaling factor is used for each index along that dimension. Set the mask to 0 to use a common scaling factor for the whole output tensor. * - scales - Constant vector of scaling factors. The following equality must hold: :math:`scales.size() = \prod\limits_{d \in mask} argument.dims[d].` .. rubric:: See also: :ref:`dnnl_primitive_attr_set_scales ` :ref:`dnnl::primitive_attr::set_output_scales ` .. index:: pair: function; get_zero_points .. _doxid-structdnnl_1_1primitive__attr_1a010dfd54aa839ca1c0da892b99963bdf: .. ref-code-block:: cpp :class: doxyrest-title-code-block void get_zero_points(int arg, int& mask, std::vector& zero_points) const Returns zero points correspondence mask and values. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - arg - Parameter argument index as passed to the :ref:`primitive::execute() ` call. * - mask - Zero points correspondence mask that defines the correspondence between the output tensor dimensions and the ``zero_points`` vector. The set i-th bit indicates that a dedicated zero point is used for each index along that dimension. Set the mask to 0 to use a common zero point for the whole output tensor. * - zero_points - Output vector of zero points. .. index:: pair: function; set_zero_points .. _doxid-structdnnl_1_1primitive__attr_1aee82deb014cf9702ceb3e725156c25a1: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_zero_points(int arg, int mask, const std::vector& zero_points) Sets zero points for primitive operations for a given memory argument. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - arg - Parameter argument index as passed to the :ref:`primitive::execute() ` call. * - mask - Zero point correspondence mask that defines the correspondence between the tensor dimensions and the ``zero_points`` vector. The set i-th bit indicates that a dedicated zero point is used for each index along that dimension. Set the mask to 0 to use a common zero point for the whole output tensor. * - zero_points - Constant vector of zero points. If the zero points are known at the time of this call, the following equality must hold: :math:`zero\_points.size() = \prod\limits_{d \in mask} argument.dims[d].` If the zero points are not known at the time of the call, this vector must contain a single :ref:`DNNL_RUNTIME_S32_VAL ` value and the zero points must be passed at execution time as an argument with index :ref:`DNNL_ARG_ATTR_ZERO_POINTS `. .. rubric:: See also: :ref:`dnnl_primitive_attr_set_zero_points ` :ref:`dnnl::primitive_attr::set_output_scales ` .. index:: pair: function; get_post_ops .. _doxid-structdnnl_1_1primitive__attr_1a05664ef63c94acbcc59e921c4a4da6b8: .. ref-code-block:: cpp :class: doxyrest-title-code-block const :ref:`post_ops` get_post_ops() const Returns post-ops previously set via :ref:`set_post_ops() `. .. rubric:: Returns: Post-ops. .. index:: pair: function; set_post_ops .. _doxid-structdnnl_1_1primitive__attr_1ac830fa9f4fcf480b494d73153ad579bf: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_post_ops(const :ref:`post_ops` ops) Sets post-ops. .. note:: There is no way to check whether the post-ops would be supported by the target primitive. Any error will be reported by the respective primitive descriptor constructor. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - ops - Post-ops object to copy post-ops from. .. index:: pair: function; set_rnn_data_qparams .. _doxid-structdnnl_1_1primitive__attr_1a39ce5aa8b06ed331d8e2158108cc8324: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_rnn_data_qparams(float scale, float shift) Sets quantization scale and shift parameters for RNN data tensors. For performance reasons, the low-precision configuration of the RNN primitives expect input activations to have the unsigned 8-bit integer data type. The scale and shift parameters are used to quantize floating-point data to unsigned integer and must be passed to the RNN primitive using attributes. The quantization formula is ``scale * data + shift``. Example usage: .. ref-code-block:: cpp // RNN parameters int l = 2, t = 2, mb = 32, sic = 32, slc = 32, dic = 32, dlc = 32; // Activations quantization parameters float scale = 63.f, shift = 64.f; :ref:`primitive_attr ` attr; // Set scale and shift for int8 quantization of activation attr.set_rnn_data_qparams(scale, shift); // Create and configure rnn op_desc vanilla_rnn_forward::desc :ref:`rnn_d `(/* arguments */); vanilla_rnn_forward::primitive_desc :ref:`rnn_d `(:ref:`rnn_d `, attr, engine); .. note:: Quantization scale and shift are common for src_layer, src_iter, dst_iter, and dst_layer. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - scale - The value to scale the data by. * - shift - The value to shift the data by. .. index:: pair: function; get_rnn_data_qparams .. _doxid-structdnnl_1_1primitive__attr_1a47d567defa762761daa2af604798d799: .. ref-code-block:: cpp :class: doxyrest-title-code-block void get_rnn_data_qparams(float& scale, float& shift) Returns the quantization scale and shift parameters for RNN data tensors. .. note:: Quantization scale and shift are common for src_layer, src_iter, dst_iter, and dst_layer. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - scale - The value to scale the data by. * - shift - The value to shift the data by. .. index:: pair: function; set_rnn_weights_qparams .. _doxid-structdnnl_1_1primitive__attr_1a61bd70f97baa628fd49b2c8b334b913e: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_rnn_weights_qparams(int mask, const std::vector& scales) Sets quantization scaling factors for RNN weights tensors. The low-precision configuration of the RNN primitives expect input weights to use the signed 8-bit integer data type. The scaling factors are used to quantize floating-point data to signed integer and must be passed to RNN primitives using attributes. .. note:: The dimension order is always native and does not depend on the actual layout used. For example, five-dimensional weights always have (l, d, i, g, o) logical dimension ordering. .. note:: Quantization scales are common for weights_layer and weights_iteration .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mask - Scaling factors correspondence mask that defines the correspondence between the output tensor dimensions and the ``scales`` vector. The set i-th bit indicates that a dedicated scaling factor should be used each index along that dimension. Set the mask to 0 to use a common scaling factor for the whole output tensor. * - scales - Constant vector of output scaling factors. The following equality must hold: :math:`scales.size() = \prod\limits_{d \in mask} weights.dims[d].` Violations can only be detected when the attributes are used to create a primitive descriptor. .. index:: pair: function; get_rnn_weights_qparams .. _doxid-structdnnl_1_1primitive__attr_1a3bbe9ac516e3aabe7dfea214210a3335: .. ref-code-block:: cpp :class: doxyrest-title-code-block void get_rnn_weights_qparams(int& mask, std::vector& scales) Returns the quantization scaling factors for RNN projection weights tensors. .. note:: The dimension order is always native and does not depend on the actual layout used. For example, five-dimensional weights always have (l, d, i, g, o) logical dimension ordering. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mask - Scaling factors correspondence mask that defines the correspondence between the output tensor dimensions and the ``scales`` vector. The set i-th bit indicates that a dedicated scaling factor should be used each index along that dimension. Set the mask to 0 to use a common scaling factor for the whole output tensor. * - scales - Constant vector of output scaling factors. The following equality must hold: :math:`scales.size() = \prod\limits_{d \in mask} weights.dims[d].` Violations can only be detected when the attributes are used to create a primitive descriptor. .. index:: pair: function; set_rnn_weights_projection_qparams .. _doxid-structdnnl_1_1primitive__attr_1a6e5a8c12f28421c249633bf2092fbe3f: .. ref-code-block:: cpp :class: doxyrest-title-code-block void set_rnn_weights_projection_qparams( int mask, const std::vector& scales ) Sets quantization scaling factors for RNN projection weights tensors. passed to RNN primitives using attributes. .. note:: The dimension order is always native and does not depend on the actual layout used. For example, five-dimensional weights always have (l, d, i, g, o) logical dimension ordering. .. note:: Quantization scales are common for weights_layer and weights_iteration .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mask - Scaling factors correspondence mask that defines the correspondence between the output tensor dimensions and the ``scales`` vector. The set i-th bit indicates that a dedicated scaling factor should be used each index along that dimension. Set the mask to 0 to use a common scaling factor for the whole output tensor. * - scales - Constant vector of output scaling factors. The following equality must hold: :math:`scales.size() = \prod\limits_{d \in mask} weights.dims[d].` Violations can only be detected when the attributes are used to create a primitive descriptor. .. index:: pair: function; get_rnn_weights_projection_qparams .. _doxid-structdnnl_1_1primitive__attr_1a53c41b29c5cd74d9485b5e753ba57f1d: .. ref-code-block:: cpp :class: doxyrest-title-code-block void get_rnn_weights_projection_qparams(int& mask, std::vector& scales) Returns the quantization scaling factors for RNN projection weights tensors. .. note:: The dimension order is always native and does not depend on the actual layout used. For example, five-dimensional weights always have (l, d, i, g, o) logical dimension ordering. .. rubric:: Parameters: .. list-table:: :widths: 20 80 * - mask - Scaling factors correspondence mask that defines the correspondence between the output tensor dimensions and the ``scales`` vector. The set i-th bit indicates that a dedicated scaling factor should be used each index along that dimension. Set the mask to 0 to use a common scaling factor for the whole output tensor. * - scales - Constant vector of output scaling factors. The following equality must hold: :math:`scales.size() = \prod\limits_{d \in mask} weights.dims[d].` Violations can only be detected when the attributes are used to create a primitive descriptor.