.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "_examples\shap\plot_main05_summaryplot.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download__examples_shap_plot_main05_summaryplot.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr__examples_shap_plot_main05_summaryplot.py:


05b. Custom using summary_plot
=====================================

This script demonstrates how to adapt the standard ``shap.summary_plot``
for visualizing the complex, three-dimensional SHAP values
(samples, timesteps, features) generated by sequential models.
It provides a powerful strategy for interpreting feature importance
both at specific points in time and across an entire sequence.

The script showcases a two-pronged visualization approach:

    - **Data Reshaping:** It begins by pivoting a tidy DataFrame into
      the wide-format matrices for SHAP values and feature values
      that are required by the plotting function.
    - **Per-Timestep Analysis:** It first iterates through each timestep,
      creating a separate summary plot that reveals the importance
      of all features at that single moment in the sequence.
    - **Per-Feature Analysis:** It then iterates through each feature,
      generating a summary plot that visualizes how the importance
      of that single feature evolves across all timesteps.

This example is essential for effectively using SHAP's most common
plot to uncover the temporal dynamics of feature contributions in
time-series and sequential data models.

.. GENERATED FROM PYTHON SOURCE LINES 28-90

.. code-block:: default
   :lineno-start: 29


    # Libraries
    import shap
    import pandas as pd

    import matplotlib.pyplot as plt


    try:
        __file__
        TERMINAL = True
    except:
        TERMINAL = False


    # ------------------------
    # Methods
    # ------------------------
    def load_shap_file():
        """Load shap file.

        .. note: The timestep does not indicate time step but matrix
                 index index. Since the matrix index for time steps
                 started in negative t=-T and ended in t=0 the
                 transformation should be taken into account.

        """
        from pathlib import Path
        # Load data
        path = Path('../../datasets/shap/')
        data = pd.read_csv(path / 'shap.csv')
        data = data.iloc[:, 1:]
        data = data.rename(columns={'timestep': 'indice'})
        data['timestep'] = data.indice - (data.indice.nunique() - 1)
        return data


    # -----------------------------------------------------
    #                       Main
    # -----------------------------------------------------
    # Load data
    # data = create_random_shap(10, 6, 4)
    data = load_shap_file()
    #data = data[data['sample'] < 100]

    shap_values = pd.pivot_table(data,
                                 values='shap_values',
                                 index=['sample', 'timestep'],
                                 columns=['features'])

    feature_values = pd.pivot_table(data,
                                    values='feature_values',
                                    index=['sample', 'timestep'],
                                    columns=['features'])

    # Show
    if TERMINAL:
        print("\nShow:")
        print(data)
        print(shap_values)
        print(feature_values)








.. GENERATED FROM PYTHON SOURCE LINES 91-92

Let's see how data looks like

.. GENERATED FROM PYTHON SOURCE LINES 92-94

.. code-block:: default
   :lineno-start: 92

    data.head(10)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>sample</th>
          <th>indice</th>
          <th>features</th>
          <th>feature_values</th>
          <th>shap_values</th>
          <th>timestep</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>0</td>
          <td>0</td>
          <td>Ward Lactate</td>
          <td>0.0</td>
          <td>0.000652</td>
          <td>-6</td>
        </tr>
        <tr>
          <th>1</th>
          <td>0</td>
          <td>0</td>
          <td>Ward Glucose</td>
          <td>0.0</td>
          <td>-0.000596</td>
          <td>-6</td>
        </tr>
        <tr>
          <th>2</th>
          <td>0</td>
          <td>0</td>
          <td>Ward sO2</td>
          <td>0.0</td>
          <td>0.000231</td>
          <td>-6</td>
        </tr>
        <tr>
          <th>3</th>
          <td>0</td>
          <td>0</td>
          <td>White blood cell count, blood</td>
          <td>0.0</td>
          <td>0.000582</td>
          <td>-6</td>
        </tr>
        <tr>
          <th>4</th>
          <td>0</td>
          <td>0</td>
          <td>Platelets</td>
          <td>0.0</td>
          <td>-0.001705</td>
          <td>-6</td>
        </tr>
        <tr>
          <th>5</th>
          <td>0</td>
          <td>0</td>
          <td>Haemoglobin</td>
          <td>0.0</td>
          <td>-0.000918</td>
          <td>-6</td>
        </tr>
        <tr>
          <th>6</th>
          <td>0</td>
          <td>0</td>
          <td>Mean cell volume, blood</td>
          <td>0.0</td>
          <td>-0.000654</td>
          <td>-6</td>
        </tr>
        <tr>
          <th>7</th>
          <td>0</td>
          <td>0</td>
          <td>Haematocrit</td>
          <td>0.0</td>
          <td>-0.000487</td>
          <td>-6</td>
        </tr>
        <tr>
          <th>8</th>
          <td>0</td>
          <td>0</td>
          <td>Mean cell haemoglobin conc, blood</td>
          <td>0.0</td>
          <td>0.000090</td>
          <td>-6</td>
        </tr>
        <tr>
          <th>9</th>
          <td>0</td>
          <td>0</td>
          <td>Mean cell haemoglobin level, blood</td>
          <td>0.0</td>
          <td>-0.000296</td>
          <td>-6</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 95-96

Let's see how shap_values looks like

.. GENERATED FROM PYTHON SOURCE LINES 96-98

.. code-block:: default
   :lineno-start: 96

    shap_values.iloc[:10, :5]






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>features</th>
          <th>Alanine Transaminase</th>
          <th>Albumin</th>
          <th>Alkaline Phosphatase</th>
          <th>Bilirubin</th>
          <th>C-Reactive Protein</th>
        </tr>
        <tr>
          <th>sample</th>
          <th>timestep</th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th rowspan="7" valign="top">0</th>
          <th>-6</th>
          <td>-0.001809</td>
          <td>0.000411</td>
          <td>0.000486</td>
          <td>0.000500</td>
          <td>0.010186</td>
        </tr>
        <tr>
          <th>-5</th>
          <td>-0.001363</td>
          <td>0.000563</td>
          <td>0.000803</td>
          <td>-0.000133</td>
          <td>0.005363</td>
        </tr>
        <tr>
          <th>-4</th>
          <td>0.001180</td>
          <td>0.000101</td>
          <td>0.000859</td>
          <td>-0.001680</td>
          <td>-0.016017</td>
        </tr>
        <tr>
          <th>-3</th>
          <td>0.004938</td>
          <td>-0.001043</td>
          <td>0.000570</td>
          <td>-0.003175</td>
          <td>-0.044723</td>
        </tr>
        <tr>
          <th>-2</th>
          <td>0.006206</td>
          <td>-0.001760</td>
          <td>0.000382</td>
          <td>-0.003976</td>
          <td>-0.062485</td>
        </tr>
        <tr>
          <th>-1</th>
          <td>-0.001391</td>
          <td>-0.004886</td>
          <td>0.002457</td>
          <td>0.010031</td>
          <td>0.056280</td>
        </tr>
        <tr>
          <th>0</th>
          <td>0.003583</td>
          <td>0.023502</td>
          <td>0.000534</td>
          <td>0.001672</td>
          <td>-0.010238</td>
        </tr>
        <tr>
          <th rowspan="3" valign="top">1</th>
          <th>-6</th>
          <td>0.000325</td>
          <td>-0.000812</td>
          <td>-0.000210</td>
          <td>-0.000157</td>
          <td>0.000971</td>
        </tr>
        <tr>
          <th>-5</th>
          <td>0.000247</td>
          <td>-0.002281</td>
          <td>-0.000301</td>
          <td>-0.000036</td>
          <td>-0.000035</td>
        </tr>
        <tr>
          <th>-4</th>
          <td>-0.000316</td>
          <td>-0.000034</td>
          <td>-0.000307</td>
          <td>0.000464</td>
          <td>-0.009348</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 99-100

Let's see how feature_values looks like

.. GENERATED FROM PYTHON SOURCE LINES 100-102

.. code-block:: default
   :lineno-start: 100

    feature_values.iloc[:10, :5]






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>features</th>
          <th>Alanine Transaminase</th>
          <th>Albumin</th>
          <th>Alkaline Phosphatase</th>
          <th>Bilirubin</th>
          <th>C-Reactive Protein</th>
        </tr>
        <tr>
          <th>sample</th>
          <th>timestep</th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th rowspan="7" valign="top">0</th>
          <th>-6</th>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>-5</th>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>-4</th>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>-3</th>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>-2</th>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>-1</th>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>0</th>
          <td>-0.982956</td>
          <td>0.237113</td>
          <td>-0.956016</td>
          <td>-0.982152</td>
          <td>-0.726284</td>
        </tr>
        <tr>
          <th rowspan="3" valign="top">1</th>
          <th>-6</th>
          <td>-0.994370</td>
          <td>-0.587629</td>
          <td>-0.956533</td>
          <td>-0.988451</td>
          <td>-0.398008</td>
        </tr>
        <tr>
          <th>-5</th>
          <td>-0.993445</td>
          <td>-0.587629</td>
          <td>-0.954463</td>
          <td>-0.990551</td>
          <td>-0.190805</td>
        </tr>
        <tr>
          <th>-4</th>
          <td>-0.994370</td>
          <td>-0.628866</td>
          <td>-0.963260</td>
          <td>-0.990551</td>
          <td>-0.307893</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 103-104

Display using ``shap.summary_plot``

.. GENERATED FROM PYTHON SOURCE LINES 104-116

.. code-block:: default
   :lineno-start: 105


    #
    # The first option is to use the ``shap`` library to plot the results.

    # Let's define/extract some useful variables.
    N = 10  # max loops filter
    TIMESTEPS = len(shap_values.index.unique(level='timestep'))  # number of timesteps
    SAMPLES = len(shap_values.index.unique(level='sample'))  # number of samples

    shap_min = data.shap_values.min()
    shap_max = data.shap_values.max()








.. GENERATED FROM PYTHON SOURCE LINES 117-118

Now, let's display the shap values for all features in each timestep.

.. GENERATED FROM PYTHON SOURCE LINES 118-138

.. code-block:: default
   :lineno-start: 121




    # For each timestep (visualise all features)
    steps = shap_values.index.get_level_values('timestep').unique()
    for i, step in enumerate(steps):
        # Get interesting indexes
        indice = shap_values.index.get_level_values('timestep') == step

        # Create auxiliary matrices
        shap_aux = shap_values.iloc[indice]
        feat_aux = feature_values.iloc[indice]

        # Display
        plt.figure()
        plt.title("Timestep: %s" % step)
        shap.summary_plot(shap_aux.to_numpy(), feat_aux, show=False)
        plt.xlim(shap_min, shap_max)





.. rst-class:: sphx-glr-horizontal


    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_001.png
         :alt: Timestep: -6
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_001.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_002.png
         :alt: Timestep: -5
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_002.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_003.png
         :alt: Timestep: -4
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_003.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_004.png
         :alt: Timestep: -3
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_004.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_005.png
         :alt: Timestep: -2
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_005.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_006.png
         :alt: Timestep: -1
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_006.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_007.png
         :alt: Timestep: 0
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_007.png
         :class: sphx-glr-multi-img





.. GENERATED FROM PYTHON SOURCE LINES 139-140

Now, let's display the shap values for all timesteps of each feature.

.. GENERATED FROM PYTHON SOURCE LINES 140-164

.. code-block:: default
   :lineno-start: 141


    # For each feature (visualise all time-steps)
    for i, f in enumerate(shap_values.columns[:N]):
        # Show
        # print('%2d. %s' % (i, f))

        # Create auxiliary matrices (select feature and reshape)
        shap_aux = shap_values.iloc[:, i] \
            .to_numpy().reshape(-1, TIMESTEPS)
        feat_aux = feature_values.iloc[:, i] \
            .to_numpy().reshape(-1, TIMESTEPS)
        feat_aux = pd.DataFrame(feat_aux,
            columns=['timestep %s' % j for j in range(-TIMESTEPS+1, 1)]
            )

        # Show
        plt.figure()
        plt.title("Feature: %s" % f)
        shap.summary_plot(shap_aux, feat_aux, sort=False, show=False, plot_type='violin')
        plt.xlim(shap_min, shap_max)
        plt.gca().invert_yaxis()

    # Show
    plt.show()



.. rst-class:: sphx-glr-horizontal


    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_008.png
         :alt: Feature: Alanine Transaminase
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_008.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_009.png
         :alt: Feature: Albumin
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_009.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_010.png
         :alt: Feature: Alkaline Phosphatase
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_010.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_011.png
         :alt: Feature: Bilirubin
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_011.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_012.png
         :alt: Feature: C-Reactive Protein
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_012.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_013.png
         :alt: Feature: Chloride
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_013.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_014.png
         :alt: Feature: Creatinine
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_014.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_015.png
         :alt: Feature: D-Dimer
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_015.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_016.png
         :alt: Feature: Eosinophils
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_016.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_017.png
         :alt: Feature: Ferritin
         :srcset: /_examples/shap/images/sphx_glr_plot_main05_summaryplot_017.png
         :class: sphx-glr-multi-img


.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    C:\Users\kelda\Desktop\repositories\github\python-spare-code\main\examples\shap\plot_main05_summaryplot.py:163: UserWarning:

    FigureCanvasAgg is non-interactive, and thus cannot be shown






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  8.284 seconds)


.. _sphx_glr_download__examples_shap_plot_main05_summaryplot.py:


.. only :: html

 .. container:: sphx-glr-footer
    :class: sphx-glr-footer-example



  .. container:: sphx-glr-download sphx-glr-download-python

     :download:`Download Python source code: plot_main05_summaryplot.py <plot_main05_summaryplot.py>`



  .. container:: sphx-glr-download sphx-glr-download-jupyter

     :download:`Download Jupyter notebook: plot_main05_summaryplot.ipynb <plot_main05_summaryplot.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_