.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "_examples/shap/plot_main06.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr__examples_shap_plot_main06.py: 06. Basic example ================= .. GENERATED FROM PYTHON SOURCE LINES 6-191 .. image-sg:: /_examples/shap/images/sphx_glr_plot_main06_001.png :alt: plot main06 :srcset: /_examples/shap/images/sphx_glr_plot_main06_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Shapes: i: (100, 2) y: (10,) x: (10, 10, 5) Data ((10, 10, 5)): [[[23 41 81 17 63] [ 2 14 27 4 31] [88 88 47 88 57] [80 81 98 28 5] [94 57 44 96 35] [34 3 23 47 63] [97 7 66 23 85] [75 9 88 67 59] [30 58 39 91 19] [64 14 8 66 86]] [[13 38 20 42 14] [57 83 7 29 31] [83 75 11 3 32] [10 54 80 1 75] [19 80 57 99 58] [38 62 20 14 9] [90 83 23 48 32] [49 52 71 92 18] [82 10 16 55 50] [ 0 13 88 31 43]] [[26 91 3 1 83] [80 19 79 55 34] [84 54 51 9 30] [84 34 95 51 56] [17 64 2 78 40] [ 6 24 67 68 49] [46 77 53 78 33] [ 6 75 17 22 64] [35 47 69 65 20] [ 5 69 54 65 45]] [[ 2 93 76 84 38] [42 20 11 96 42] [43 12 94 30 14] [65 66 66 44 86] [65 22 10 8 68] [38 29 1 46 56] [91 84 11 44 22] [ 6 73 18 64 12] [39 61 52 77 98] [ 9 80 69 94 42]] [[97 57 57 4 72] [85 22 24 61 34] [87 94 59 27 19] [36 76 56 78 26] [14 9 58 30 1] [59 71 6 98 88] [35 31 79 44 70] [30 89 88 18 49] [43 11 14 53 51] [ 0 66 42 26 46]] [[69 97 35 71 45] [38 32 34 29 24] [80 93 3 30 5] [ 7 25 63 50 62] [74 49 16 52 10] [94 72 6 22 82] [43 19 32 10 51] [ 2 2 12 77 99] [82 8 93 61 44] [13 87 67 86 3]] [[74 73 64 57 33] [46 95 99 60 87] [16 1 45 29 27] [44 9 89 41 58] [48 60 65 30 25] [80 26 57 76 47] [47 45 24 89 34] [67 30 7 20 79] [33 92 28 62 97] [24 66 83 55 46]] [[58 91 23 65 86] [15 10 57 44 62] [ 6 80 51 45 58] [48 22 28 82 29] [25 56 14 14 76] [89 64 46 63 50] [ 1 51 18 66 98] [26 1 89 74 94] [28 32 85 2 48] [ 7 17 90 19 80]] [[61 66 98 61 96] [60 71 80 98 97] [93 38 12 32 6] [52 18 58 18 70] [80 65 32 98 59] [ 3 97 86 38 12] [49 97 2 43 27] [82 82 68 17 19] [72 33 91 97 96] [47 92 21 99 95]] [[53 7 86 30 2] [48 75 21 15 23] [17 36 13 59 24] [30 21 76 51 34] [87 46 82 99 62] [77 60 33 11 21] [34 68 76 67 30] [ 8 48 64 73 84] [16 93 9 99 94] [85 29 79 55 95]]] DataFrame (2D) id t f0 f1 f2 f3 f4 0 0 0 23 41 81 17 63 1 0 1 2 14 27 4 31 2 0 2 88 88 47 88 57 3 0 3 80 81 98 28 5 4 0 4 94 57 44 96 35 .. .. .. .. .. .. .. .. 95 9 5 77 60 33 11 21 96 9 6 34 68 76 67 30 97 9 7 8 48 64 73 84 98 9 8 16 93 9 99 94 99 9 9 85 29 79 55 95 [100 rows x 7 columns] Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= lstm (LSTM) (None, 64) 17920 dense (Dense) (None, 64) 4160 dense_1 (Dense) (None, 1) 65 ================================================================= Total params: 22,145 Trainable params: 22,145 Non-trainable params: 0 _________________________________________________________________ None Train on 10 samples Epoch 1/16 10/10 [==============================] - ETA: 0s - loss: 0.7290 - acc: 0.3000 10/10 [==============================] - 0s 16ms/sample - loss: 0.7290 - acc: 0.3000 Epoch 2/16 10/10 [==============================] - ETA: 0s - loss: 0.6919 - acc: 0.4000 10/10 [==============================] - 0s 304us/sample - loss: 0.6919 - acc: 0.4000 Epoch 3/16 10/10 [==============================] - ETA: 0s - loss: 0.6580 - acc: 0.5000 10/10 [==============================] - 0s 256us/sample - loss: 0.6580 - acc: 0.5000 Epoch 4/16 10/10 [==============================] - ETA: 0s - loss: 0.6270 - acc: 0.6000 10/10 [==============================] - 0s 256us/sample - loss: 0.6270 - acc: 0.6000 Epoch 5/16 10/10 [==============================] - ETA: 0s - loss: 0.6002 - acc: 0.6000 10/10 [==============================] - 0s 262us/sample - loss: 0.6002 - acc: 0.6000 Epoch 6/16 10/10 [==============================] - ETA: 0s - loss: 0.5756 - acc: 0.6000 10/10 [==============================] - 0s 276us/sample - loss: 0.5756 - acc: 0.6000 Epoch 7/16 10/10 [==============================] - ETA: 0s - loss: 0.5517 - acc: 0.6000 10/10 [==============================] - 0s 274us/sample - loss: 0.5517 - acc: 0.6000 Epoch 8/16 10/10 [==============================] - ETA: 0s - loss: 0.5298 - acc: 0.6000 10/10 [==============================] - 0s 265us/sample - loss: 0.5298 - acc: 0.6000 Epoch 9/16 10/10 [==============================] - ETA: 0s - loss: 0.5094 - acc: 0.8000 10/10 [==============================] - 0s 281us/sample - loss: 0.5094 - acc: 0.8000 Epoch 10/16 10/10 [==============================] - ETA: 0s - loss: 0.4905 - acc: 0.8000 10/10 [==============================] - 0s 271us/sample - loss: 0.4905 - acc: 0.8000 Epoch 11/16 10/10 [==============================] - ETA: 0s - loss: 0.4724 - acc: 0.8000 10/10 [==============================] - 0s 261us/sample - loss: 0.4724 - acc: 0.8000 Epoch 12/16 10/10 [==============================] - ETA: 0s - loss: 0.4546 - acc: 0.8000 10/10 [==============================] - 0s 279us/sample - loss: 0.4546 - acc: 0.8000 Epoch 13/16 10/10 [==============================] - ETA: 0s - loss: 0.4367 - acc: 0.9000 10/10 [==============================] - 0s 271us/sample - loss: 0.4367 - acc: 0.9000 Epoch 14/16 10/10 [==============================] - ETA: 0s - loss: 0.4193 - acc: 0.9000 10/10 [==============================] - 0s 271us/sample - loss: 0.4193 - acc: 0.9000 Epoch 15/16 10/10 [==============================] - ETA: 0s - loss: 0.4028 - acc: 0.9000 10/10 [==============================] - 0s 265us/sample - loss: 0.4028 - acc: 0.9000 Epoch 16/16 10/10 [==============================] - ETA: 0s - loss: 0.3865 - acc: 0.9000 10/10 [==============================] - 0s 275us/sample - loss: 0.3865 - acc: 0.9000 (10, 10, 5) timestep 0 timestep 1 timestep 2 timestep 3 timestep 4 timestep 5 timestep 6 timestep 7 timestep 8 timestep 9 0 41 14 88 81 57 3 7 9 58 14 1 38 83 75 54 80 62 83 52 10 13 2 91 19 54 34 64 24 77 75 47 69 3 93 20 12 66 22 29 84 73 61 80 4 57 22 94 76 9 71 31 89 11 66 5 97 32 93 25 49 72 19 2 8 87 6 73 95 1 9 60 26 45 30 92 66 7 91 10 80 22 56 64 51 1 32 17 8 66 71 38 18 65 97 97 82 33 92 9 7 75 36 21 46 60 68 48 93 29 '\n#y_pred = model.predict(x[:3, :, :])\n#print(y_pred)\n\n#background = x[np.random.choice(x.shape[0], 10, replace=False)]\nmasker = shap.maskers.Independent(data=x)\n# Get generic explainer\n#explainer = shap.KernelExplainer(model, background)\nexplainer = shap.KernelExplainer(model.predict, x, masker=masker)\n\n# Show kernel type\nprint("\nKernel type: %s" % type(explainer))\n\n# Get shap values\nshap_values = explainer.shap_values(x)\n\nprint(shap_values)\n' | .. code-block:: default :lineno-start: 6 # Libraries import shap import numpy as np import pandas as pd import tensorflow as tf tf.compat.v1.disable_eager_execution() tf.compat.v1.disable_v2_behavior() # -------------------------------------------- # Create data # -------------------------------------------- # Constants SAMPLES = 10 TIMESTEPS = 10 FEATURES = 5 # .. note: Either perform a pre-processing step such as # normalization or generate the features within # the appropriate interval. # Create dataset x = np.random.randint(low=0, high=100, size=(SAMPLES, TIMESTEPS, FEATURES)) y = np.random.randint(low=0, high=2, size=SAMPLES).astype(float) i = np.vstack(np.dstack(np.indices((SAMPLES, TIMESTEPS)))) # Create DataFrame df = pd.DataFrame( data=np.hstack((i, x.reshape((-1,FEATURES)))), columns=['id', 't'] + ['f%s'%j for j in range(FEATURES)] ) # Show print("Shapes:") print("i: %s" % str(i.shape)) print("y: %s" % str(y.shape)) print("x: %s" % str(x.shape)) print("\nData (%s):" % str(x.shape)) print(x) print("\nDataFrame (2D)") print(df) # -------------------------------------------- # Model # -------------------------------------------- # Libraries from tensorflow.keras.layers import Input from tensorflow.keras.layers import Dropout from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense from tensorflow.keras.layers import LSTM from tensorflow.keras.layers import Embedding from tensorflow.keras.preprocessing import sequence # Create model model = Sequential() #model.add(Input(shape=(None, FEATURES))) model.add( LSTM( units=64, return_sequences=False, input_shape=(TIMESTEPS, FEATURES) )) #model.add(Dropout(0.2)) model.add(Dense(64, activation='relu')) model.add(Dense(1, activation='sigmoid')) model.compile( loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'] ) model.run_eagerly = False # Load pre-trained weights # Display model summary print(model.summary()) model.save('outputs/model.h5') # Fit model.fit(x, y, epochs=16, batch_size=64) # -------------------------------------------- # Compute and display SHAP values # -------------------------------------------- # https://github.com/slundberg/shap/blob/master/shap/plots/_beeswarm.py # Use the training data for deep explainer => can use fewer instances explainer = shap.DeepExplainer(model, x) # explain the the testing instances (can use fewer instanaces) # explaining each prediction requires 2 * background dataset size runs shap_values = explainer.shap_values(x) # init the JS visualization code shap.initjs() print(shap_values[0].shape) #shap_values = explainer(x) """ shap.plots.beeswarm(shap_values, max_display=12, order=shap.Explanation.abs.mean(0)) import matplotlib.pyplot as plt plt.show() import sys sys.exit() """ shap_values_2D = shap_values[0].reshape(-1,x.shape[-1]) x_2D = pd.DataFrame( data=x.reshape(-1,x.shape[-1]), columns=['f%s'%j for j in range(x.shape[-1])] ) ## SHAP for each time step NUM_STEPS = x.shape[1] NUM_FEATURES = x.shape[-1] len_test_set = x_2D.shape[0] """ # step = 0 for step in range(NUM_STEPS): indice = [i for i in list(range(len_test_set)) if i%NUM_STEPS == step] shap_values_2D_step = shap_values_2D[indice] x_test_2d_step = x_2D.iloc[indice] print("_______ time step {} ___________".format(step)) #shap.summary_plot(shap_values_2D_step, x_test_2d_step, plot_type="bar") shap.summary_plot(shap_values_2D_step, x_test_2d_step) print("\n") """ shap_values_2D_step = shap_values_2D[:, 1].reshape(-1, x.shape[1]) x_test_2d_step = x_2D.iloc[:, 1].to_numpy().reshape(-1, x.shape[1]) x_test_2d_step = pd.DataFrame( x_test_2d_step, columns=['timestep %s'%j for j in range(x.shape[1])] ) print(x_test_2d_step) shap.summary_plot(shap_values_2D_step, x_test_2d_step, sort=False) """ for step in range(NUM_STEPS): indice = [i for i in list(range(len_test_set)) if i%NUM_STEPS == step] shap_values_2D_step = shap_values_2D[indice] x_test_2d_step = x_2D.iloc[indice] print("_______ time step {} ___________".format(step)) #shap.summary_plot(shap_values_2D_step, x_test_2d_step, plot_type="bar") shap.summary_plot(shap_values_2D_step, x_test_2d_step) print("\n") """ import matplotlib.pyplot as plt plt.show() """ #y_pred = model.predict(x[:3, :, :]) #print(y_pred) #background = x[np.random.choice(x.shape[0], 10, replace=False)] masker = shap.maskers.Independent(data=x) # Get generic explainer #explainer = shap.KernelExplainer(model, background) explainer = shap.KernelExplainer(model.predict, x, masker=masker) # Show kernel type print("\nKernel type: %s" % type(explainer)) # Get shap values shap_values = explainer.shap_values(x) print(shap_values) """ .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 8.899 seconds) .. _sphx_glr_download__examples_shap_plot_main06.py: .. only :: html .. container:: sphx-glr-footer :class: sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_main06.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_main06.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_