.. index:: pair: page; Persistent Cache
.. _doxid-dev_guide_persistent_cache:

Persistent Cache
================

:target:`doxid-dev_guide_persistent_cache_1md_doc_advanced_persistent_cache`

oneDNN provides :ref:`primitive cache <doxid-dev_guide_primitive_cache>` to reduce primitive creation overhead. However, the primitive cache has no effect when a primitive is created for the first time. For some applications it can be critical to reduce that overhead.

oneDNN provides an API that can be used to create a persistent cache. User can use that API to obtain a cache blob ID from a primitive descriptor and a cache blob from a primitive to use them as a key and value respectively. The cache blob contains data that can be used to speed up primitive creation.

.. note:: 

   Content and size of the cache blob ID and cache blob objects are not specified.
   
   


* The cache blob ID can be obtained via :ref:`dnnl::primitive_desc_base::get_cache_blob_id <doxid-structdnnl_1_1primitive__desc__base_1a5e24a01371a9e3168595c2312f4e3c6e>`

* The cache blob can be obtained via :ref:`dnnl::primitive::get_cache_blob <doxid-group__dnnl__api__primitives__common_1ga8bf59b36c745ee8eaec9d0dd22e266e9>`

* Each primitive class provides a constructor that takes the cache blob along with the primitive descriptor.

.. note:: 

   oneDNN version and git commit hash (:ref:`dnnl_version_t::hash <doxid-structdnnl__version__t_1a1c2a7e8b3f26ed376b537dd511feccad>`) affect equality of the cache blob IDs. That is, the cache blob ID obtained from a primitive descriptor will be different for different oneDNN versions and git commit hashes.
   
   

.. warning:: 

   The git commit hash may not be available if the git package was not found during a CMake call. In this case, the cache blob ID will be the same for different hashes. This may result in fetching a wrong cache blob from persistent cache.



.. _doxid-dev_guide_persistent_cache_1autotoc_md18:

Relation to Primitive Cache
~~~~~~~~~~~~~~~~~~~~~~~~~~~

In the case when a primitive is created from a cache blob and the identical primitive is present in the primitive cache the one from primitive cache will be returned to the user, and the given cache blob will not be used. Otherwise, the cache blob will be used to speed up the primitive creation. The information about how the primitive was created (``cache_miss``, ``cache_hit`` or ``from_cache_blob``) is part of the verbose output for verbose level 2 (:ref:`Verbose Mode <doxid-dev_guide_verbose>`).





.. _doxid-dev_guide_persistent_cache_1autotoc_md19:

API Usage Example
~~~~~~~~~~~~~~~~~

The following pseudo-code demonstrates a simple example of persistent cache implementation using the oneDNN API:

.. ref-code-block:: cpp

	using namespace :ref:`dnnl <doxid-namespacednnl>`;
	
	{
	    :ref:`convolution_forward::primitive_desc <doxid-structdnnl_1_1convolution__forward_1_1primitive__desc>` conv_pd(desc, attr, :ref:`engine <doxid-structdnnl_1_1engine>`);
	    :ref:`convolution_forward <doxid-structdnnl_1_1convolution__forward>` conv(conv_pd);
	
	    std::vector<uint8_t> key = conv_pd.get_cache_blob_id();
	    std::vector<uint8_t> value = conv.get_cache_blob();
	    store_cache_blob_on_disk(key, value);
	}
	
	{
	    :ref:`convolution_forward::primitive_desc <doxid-structdnnl_1_1convolution__forward_1_1primitive__desc>` conv_pd(desc, attr, :ref:`engine <doxid-structdnnl_1_1engine>`);
	    std::vector<uint8_t> key = conv_pd.get_cache_blob_id();
	    std::vector<uint8_t> value = load_cache_blob_from_disk(key);
	    :ref:`convolution_forward <doxid-structdnnl_1_1convolution__forward>` conv_from_cache_blob(conv_pd, value);
	}





.. _doxid-dev_guide_persistent_cache_1autotoc_md20:

Limitations
~~~~~~~~~~~

The API is implemented for the OpenCL runtime only. For CPU engine kind and other runtimes the library will return :ref:`dnnl_unimplemented <doxid-group__dnnl__api__utils_1ggad24f9ded06e34d3ee71e7fc4b408d57aa3a8579e8afc4e23344cd3115b0e81de1>` in the case of the C API or throw a corresponding :ref:`dnnl::error <doxid-structdnnl_1_1error>` exception in the case of the C++ API.