[GNA] NCHW to NHWC layout transformations (#18120)

* Squashed commit of the following: GNAConv+ MaxPool

commit 51e7254eab
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Tue Apr 18 18:09:46 2023 +0200

    reset json schema

commit 6d87c40703
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Tue Apr 18 18:02:53 2023 +0200

    Exclude wrong changes

commit 30dd9aef86
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Tue Apr 18 17:56:54 2023 +0200

    clang format

commit dad7151b37
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Tue Apr 18 17:47:17 2023 +0200

    Init commit

* Squashed commit of the following: GatherSinking binary elementwise

commit 960f7ebaed
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Feb 21 13:36:35 2023 +0100

    fix year

commit 860ae33f38
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Feb 21 13:29:05 2023 +0100

    add docs

commit bd35e57c60
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Mon Feb 20 18:58:55 2023 +0100

    cleanup

commit 3fed498f03
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Mon Feb 20 18:52:49 2023 +0100

    clang fixes + remove unneeded functions

commit 4417a13bad
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Mon Feb 20 18:50:03 2023 +0100

    fix unit tests execution

commit 48f20927af
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Mon Feb 20 18:33:03 2023 +0100

    initial

* Squashed commit of the following: common debug merge

commit 9cdddb337972c2ebecb3ed81f2613b96ae034538
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Fri May 12 14:05:34 2023 +0200

    clnag format

commit f093baa7708bd9e9527277337e52104b4e82744d
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Apr 19 15:27:58 2023 +0200

    fix ts_concat ts_split

commit 1c8baba5bb4c1533f4470505fe5a0dce6e4bc45e
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Apr 19 13:26:30 2023 +0200

    fix ts_split

commit 043af9aad51b218c8e27150516e3824cea682ca3
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Apr 18 19:39:15 2023 +0200

    remove debug

commit 7c03af67f018d5eea2c39d01bf2d68b815cde390
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Apr 18 19:37:34 2023 +0200

    fix ts_concat

commit dc2f00a1e77427b0ae243f5ef7606eac4cdf8051
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Apr 18 13:15:50 2023 +0200

    implemented ts_split through gather

commit 311045f78d357b4f1129a4545da9092435313664
Merge: 06fee24288 7ca64ec9f7
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Mon Apr 17 19:19:05 2023 +0200

    Merge branch 'gna_layout_debug_fix' into gna_layout_debug

commit 7ca64ec9f704c2228808212e267fb387c08e4921
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Mon Apr 17 19:06:44 2023 +0200

    fix

commit 81c6217c68fe6f945ab83ae8b6ed64d23415a62e
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Mon Apr 17 18:29:32 2023 +0200

    fix

commit 06fee24288be846209e87fbd40aab7cad2775559
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Mon Apr 17 16:03:34 2023 +0200

    fix RemoveOutputsProcessing name to emplace

commit 83344b2723b4db919862e7519369593710d06de9
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Mon Apr 17 10:35:59 2023 +0200

    fix

commit cc7d6bdea26df1d2e3195bf82e62193442723995
Merge: d916fa40f9 83d51b5fe4
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Thu Apr 13 13:32:44 2023 +0200

    Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

commit d916fa40f98d5d253e445929319f06b638d2f430
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Thu Apr 13 13:32:02 2023 +0200

    remove debug

commit f8b1fac6f94577059f729ac7c31594b4f93ba4ae
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Thu Apr 13 13:31:12 2023 +0200

    fix + remove debug

commit 710d004eda3063374b99c3a78a37f7af2b84d1e8
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Thu Apr 13 11:54:36 2023 +0200

    fix

commit 83d51b5fe485d96c76bc77ace671491c5bfb1223
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Wed Apr 12 19:06:15 2023 +0200

    Pattern Reshape-Transpose

commit 3e33e51edff7762112f00d29281a59a6cb16ea83
Merge: a5042ebce7 74c68cbf4f
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Wed Apr 12 16:56:24 2023 +0200

    Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

commit a5042ebce79b0fc6a5db749166a4784aec9f0d2a
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Wed Apr 12 16:56:00 2023 +0200

    Reverted check of concat with interleaved inputs

commit 74c68cbf4fd1206d5e67326a4cf0dc290fbeb574
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Apr 12 16:53:17 2023 +0200

    use ts_fuse instead of general after all

commit 49ac734247ef6f73ce2ca096f1bd03c027f3c338
Merge: 79d23d893f 1dbea2cb3c
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Wed Apr 12 15:02:35 2023 +0200

    Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

commit 79d23d893fd585f253eec8d3e10198e5b75ed2da
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Wed Apr 12 15:02:08 2023 +0200

    Fixed ConvertMatmulToPointwiseConvWithFqNeg

commit 1dbea2cb3c948d2703a2e97957b25ebd2bd30db6
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Apr 12 14:29:22 2023 +0200

    fix pipeline

commit d73effb2c1905754a9f8636e0a762885224ca44f
Merge: e2b5440ad1 d5b7a7c984
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Apr 12 14:01:28 2023 +0200

    Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

commit e2b5440ad11eea5aa456146fa12a5f07223c2f43
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Apr 12 14:01:25 2023 +0200

    fix ts_split bug when we some split outputs are not connected

commit d5b7a7c98477df74fd6e1a8481c8bc513854001b
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Wed Apr 12 13:57:25 2023 +0200

    Corrected limit conditions

commit 7de41f2072420f3d1cb1f933991853ba787a8f42
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Apr 12 13:04:42 2023 +0200

    remove debug

commit ad2584aa23265c3cdb6370d1b87a8c0af89c17ad
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Apr 11 18:48:49 2023 +0200

    use TSConcatForward TSSplitBackward

commit 7c9fabb239415e3f1af01058aaca8b9bc63d6c26
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Apr 11 18:33:03 2023 +0200

    fix ts_split

commit da1488d21ccb649e75d0e61d424bf8e8f5d9e8f2
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Apr 11 14:19:26 2023 +0200

    fixed ts_concat

commit 2014b727ed387ca2035de5ec144f1beb67c32dd9
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Thu Apr 6 11:42:17 2023 +0200

    add ts_split

commit 7316a17131a6cb0a8c25061a6b12afdee004661d
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Thu Apr 6 11:42:00 2023 +0200

    remove debug

commit 05c9b9524781f2017d6c5f88b793c210dfd2ab16
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Thu Apr 6 11:41:10 2023 +0200

    remove debug, turn off new transformations

commit 8962424b907f7fec2ddfe32912f8da77c13461e4
Merge: f98adb6dc4 99090c2090
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Apr 5 16:50:41 2023 +0200

    Merge branch 'gna_layout_debug' into gna_layout_debug_concat_split

commit f98adb6dc40b4e368e97b58cc28ad3554fb54923
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Apr 5 16:50:17 2023 +0200

    remove debug

commit 9dddf3ada89b65da3191f647bf2f17af6d8a092a
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Apr 5 16:44:34 2023 +0200

    fix ts_concat sigsegv

commit 190010f4dfca82bb4b5a3e045e6015b8d1db8491
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Apr 5 13:15:53 2023 +0200

    fix problem with transpose backward sinking after gna::ts_concat

commit 99090c2090e3a121241a54a5b8da290ba5272b11
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Apr 5 12:52:13 2023 +0200

    fix bug GatherSinkingTransposeReshapeForward map:at exception

commit a6f65aae2d94a63f60440d9fcdeeb31af548b055
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Apr 5 12:51:19 2023 +0200

    add ts_concat

commit d9497eadad5c58ec9218692a08da54c3d7fd3d40
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Apr 4 13:40:51 2023 +0200

    add code from Misha

commit a5fe706b2bf92bcc1f3f8c435663551a0770188c
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Tue Apr 4 13:00:47 2023 +0200

    TS concat callback draft

commit 8f244188c4d0ceef11850e118fd0ad23fcdac0b8
Merge: 5591091e05 a5fe706b2b
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Apr 4 12:57:23 2023 +0200

    Merge branch 'gna_layout_debug' into gna_layout_debug_concat_split

commit 5591091e05a32b8e3566ec70490cbd373f827910
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Apr 4 12:56:48 2023 +0200

    fix

commit 7f9eb3c02c57cbef5b32efa5d61cf7082ca6b621
Merge: a98a353f00 9190b8ccff
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Apr 4 11:20:26 2023 +0200

    merge

commit a98a353f00cdb1650edfe0f5f24a10111c0319b7
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Apr 4 11:19:43 2023 +0200

    fix

commit a8ca19419f7b1e16064c8cb53ed836370da8af5e
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Tue Apr 4 11:14:44 2023 +0200

    Fixed weights reducer

commit 9190b8ccffd1a125cb97d0e339f0820f1c565d3c
Merge: 279f76287e 4ce59b6885
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Apr 4 10:46:07 2023 +0200

    Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

commit 279f76287eb7e656c3dd743a228839c16470057a
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Apr 4 10:46:03 2023 +0200

    use transformation_callback in ts_concat

commit 3fabc31ed9462f86ae173aba927caf9c9c3fb539
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Apr 4 10:44:42 2023 +0200

    fix

commit d942074be54a99147c893eb78e878cc4428be5ad
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Mon Apr 3 18:29:02 2023 +0200

    debug

commit 4ce59b68851f71a049b34d16008a6ed82cc91d2b
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Mon Apr 3 17:49:45 2023 +0200

    Skipped supported transposes in Gather transform

commit 3d57040037493aa61ffb1d159e76567ca493dd33
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Mon Apr 3 12:16:58 2023 +0200

    Extended FQ detection

commit bcc317e80d1f9ed6d6f6aa5c77ff5e58e31e84db
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Mon Apr 3 10:11:48 2023 +0200

    Gather sinking with FQ Backward

commit b1dc3c44f3cca1035ce4bd3ff05f1f78a7add1ce
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Fri Mar 31 10:48:38 2023 +0200

    Fixed typo

commit 76b2e162156e20a1ade9afc893e9874b8c66284d
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Tue Mar 28 15:08:24 2023 +0200

    FQ fix

commit 99df8e6164cad78a25e34ea752da9264c7536188
Merge: 07cdff9a44 8894dfc88b
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Mon Mar 27 18:03:54 2023 +0200

    Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

commit 07cdff9a44585026bbd58fc1c33b1e67d47bd225
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Mon Mar 27 18:03:51 2023 +0200

    add GatherSinkingMatmulBackward

commit 8894dfc88b22736b1157cd75e6b48ad14a4a1137
Merge: 0e0dcc6a53 67a4035902
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Fri Mar 24 14:28:39 2023 +0100

    Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

commit 67a4035902
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Fri Mar 24 13:59:20 2023 +0100

    remove debug

commit 04a715267a
Merge: 7b95c90df8 1163b926ee
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Fri Mar 24 13:56:05 2023 +0100

    Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

commit 7b95c90df8
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Fri Mar 24 13:54:49 2023 +0100

    fix gather_sinking_matmul

commit 0e0dcc6a53ce660b10c27d7e1d48de260e87e127
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Fri Mar 24 10:43:07 2023 +0100

    Disabled ReshapeTransposeSubstitute

commit daaeeecfac3ca7b1763279e62455d007245bcdf8
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Fri Mar 24 10:41:56 2023 +0100

    Fixed squeeze helper

commit 1163b926ee
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Thu Mar 23 16:09:59 2023 +0100

    Corrected Concat checks

commit 06d6fbf0e8
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Thu Mar 23 15:28:00 2023 +0100

    initial

commit 1ca78f643e
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Thu Mar 23 15:27:21 2023 +0100

    debug

commit 3c5aa8c53d
Merge: 01de25e3c6 aaba7740d4
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Thu Mar 23 15:13:02 2023 +0100

    Merge branch 'itikhono/ts/slice' into gna_layout_debug

commit 01de25e3c6
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Thu Mar 23 13:35:35 2023 +0100

    detect matmuls

commit 45c1f800f3
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Thu Mar 23 12:34:28 2023 +0100

    Removed do_rotate flag processing

commit aaba7740d4
Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
Date:   Thu Mar 23 10:01:58 2023 +0000

    codestyle

commit 979d5cc669
Merge: 67c1b9daad a66868b463
Author: Ivan <ivan.tikhonov@intel.com>
Date:   Thu Mar 23 17:48:05 2023 +0400

    Merge branch 'itikhono/ts/slice' of https://github.com/itikhono/openvino into itikhono/ts/slice

commit 67c1b9daad
Author: Ivan <ivan.tikhonov@intel.com>
Date:   Thu Mar 23 17:47:50 2023 +0400

    resolve review comments

commit 8b81867d18
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Thu Mar 23 10:15:42 2023 +0100

    Reverted GetConvData helpers

commit d78f7e7fc6
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Mar 22 17:29:02 2023 +0100

    add gather sinking split

commit e1ff589d3e
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Mar 22 17:07:12 2023 +0100

    ad Reshape to supported gather sinking nodes

commit 894defdcc9
Merge: 31d7af368e 4fd047deec
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Mar 22 16:00:51 2023 +0100

    Merge branch 'gna_layout_debug' into gather_sinking_reshape

commit 31d7af368e
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Wed Mar 22 16:00:00 2023 +0100

    add gather_sinking_reshape

commit 4fd047deec
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Wed Mar 22 15:29:59 2023 +0100

    Fixed squeeze condition

commit d105cfcc68
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Wed Mar 22 14:55:10 2023 +0100

    Fixed Gather indexes

commit a66868b463
Merge: bf9bc8628b b70e56d110
Author: Ivan Tikhonov <ivan.tikhonov@intel.com>
Date:   Wed Mar 22 01:53:10 2023 +0400

    Merge branch 'master' into itikhono/ts/slice

commit bf9bc8628b
Merge: 31bd608ac4 3d508c7b36
Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
Date:   Tue Mar 21 21:41:57 2023 +0000

    Merge branch 'itikhono/ts/slice' of https://github.com/itikhono/openvino into itikhono/ts/slice

commit 31bd608ac4
Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
Date:   Tue Mar 21 21:41:37 2023 +0000

    fix TSSqueeze transformation

commit 02abf9b1f0
Merge: 4481041f86 5c249d98d9
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Mar 21 19:48:06 2023 +0100

    Merge branch 'transpose_sinking_fakequantize' into gna_layout_debug

commit 5c249d98d9
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Mar 21 19:47:27 2023 +0100

    fix

commit cdfd77a415
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Mar 21 19:43:43 2023 +0100

    rebase fixes

commit e4207c4d6b
Author: Ivan <ivan.tikhonov@intel.com>
Date:   Tue Mar 21 20:21:42 2023 +0400

    fix unit tests, revert changes for TSSlice transformation

commit 4481041f86
Merge: eac5d4beb4 b1099c1c40
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Tue Mar 21 16:16:56 2023 +0100

    Merge branch 'transpose_sinking_fakequantize' into gna_layout_debug

commit b1099c1c40
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Thu Mar 2 14:38:43 2023 +0100

    initial

commit 0f17c5f714
Merge: 981e8ad3c0 db220f11ee
Author: Ivan <ivan.tikhonov@intel.com>
Date:   Tue Mar 21 18:24:08 2023 +0400

    Merge branch 'itikhono/ts/slice' of https://github.com/itikhono/openvino into itikhono/ts/slice

commit 3d508c7b36
Merge: 9720a28fa6 0893efe073
Author: Ivan Tikhonov <ivan.tikhonov@intel.com>
Date:   Tue Mar 21 17:07:42 2023 +0400

    Merge branch 'master' into itikhono/ts/slice

commit 9720a28fa6
Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
Date:   Tue Mar 21 12:35:23 2023 +0000

    codestyle

commit eac5d4beb4
Merge: faf2f253c6 db220f11ee
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Tue Mar 21 12:45:59 2023 +0100

    Merge branch 'itikhono/ts/slice' into gna_layout_debug

    # Conflicts:
    #	src/common/transformations/src/transformations/transpose_sinking/ts_slice.cpp
    #	src/common/transformations/src/transformations/transpose_sinking/ts_squeeze.cpp
    #	src/common/transformations/src/transformations/transpose_sinking/ts_unsqueeze.cpp

commit faf2f253c6
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Tue Mar 21 12:12:18 2023 +0100

    Squeeze shapes when detect gather

commit 7b1bbf77ee
Author: Evgeny Kotov <evgeny.kotov@intel.com>
Date:   Mon Mar 20 19:48:56 2023 +0100

    fix /gather_sinking_transpose_reshape

commit e6650cfd1c
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Tue Mar 21 12:06:01 2023 +0100

    Corrected Convolution checks

commit 83f4428f48
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Mon Mar 20 18:01:35 2023 +0100

    Corrected reshape paterns

commit 0552dfe537
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Sun Mar 19 18:39:01 2023 +0100

    Merged the latest transpose sinking changes

commit 56a5e74bf4
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Sun Mar 19 18:21:06 2023 +0100

    Revert "Squashed commit of the following:"

    This reverts commit 227274e70696def4a718e65799edb3b50c643c65.

commit fd8effeef5
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Sun Mar 19 18:17:25 2023 +0100

    Squashed commit of the following:

    commit f1b49907a34c94ca91fad643e8949d2ae38b2f44
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Fri Mar 17 15:44:24 2023 +0100

        Removed unnecessary transposes

    commit e57f26cf775d74a61d6e505f785211bdf2577405
    Author: Ilya Lavrenov <ilya.lavrenov@intel.com>
    Date:   Thu Mar 9 12:32:02 2023 +0400

        Fixed PT FE compilation with clang on macOS (#16173)

    commit 3461b52d79f7895df554e7ee023ae7a6448e7b42
    Author: Mateusz Bencer <mateusz.bencer@intel.com>
    Date:   Thu Mar 9 07:22:06 2023 +0100

        [ONNX FE] Improved a method of operators registration (#15990)

        * initial version of implementation

        * styles applied

        * fixed and registration

        * add more unit tests

        * fixed and in legacy opset

        * review remarks

        * refactor of version name range

    commit 1044df5afa63a052750647f57202ce8b7da1906b
    Author: Jade Cho <jade.cho@intel.com>
    Date:   Thu Mar 9 14:35:31 2023 +0900

        [GPU] Fix stable diffusion failure (#16052)

        * [dGPU] Enable stable diffusion

        + Prevent to fuse swish into oneDNN reorder.
        + Makes concat explicitly if batch size is greater than 1 and the siblings are oneDNN impl.

    commit da1c5cb763b301b050f3f52f362e172c1a42758b
    Author: Andrew Kwangwoong Park <andrew.park@intel.com>
    Date:   Thu Mar 9 08:36:43 2023 +0900

        [GPU] Added shape agnostic Pad kernel implementation (#16160)

        Signed-off-by: Andrew Park <andrew.park@intel.com>

    commit 03b96e1d3b41ddca01de7f4117836e5708bd0b7f
    Merge: ca9e0f6d41 c3ecce0a7e
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Fri Mar 17 11:18:09 2023 +0100

        Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

    commit ca9e0f6d41b95e1653d07c03684efb170d856429
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Fri Mar 17 11:14:36 2023 +0100

        Fixed validator of 1D convolution

    commit c3ecce0a7ea2944b42290c21535b4600a406a1ce
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Thu Mar 16 19:03:10 2023 +0100

        fix conditions on mvn and slice

    commit a6f920ed468943a1c669fdff8b8b0217281ee302
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Thu Mar 16 17:43:35 2023 +0100

        fix tests with Slice

    commit 64b1fa67f5a0ac821d1af5cb132cfd0575041aed
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Thu Mar 16 17:33:38 2023 +0100

        fix debug

    commit ca122e890e1f1590655e9e776d4c858dc3433890
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Mar 15 16:14:49 2023 +0100

        move RemovePermutationsNHWCToNCHWPass under is_ngraph_passes_used

    commit ffa21d987b5f7bc7d374e825ffa6bf4dff7679eb
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Wed Mar 15 13:02:12 2023 +0100

        Removed ConvIE code

    commit 23ca385ba601a06985ea218e53c5a05cdd3b9b87
    Merge: 2acd7c4bff 90b37d771b
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Wed Mar 15 11:47:57 2023 +0100

        Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

    commit 2acd7c4bff4422c95f6b8fb511287110ceb8e99d
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Wed Mar 15 11:45:46 2023 +0100

        Disabled sinking for networks without Convolutions

    commit 90b37d771b9cb9df8d0e1a5d72cb7acc8d01634f
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Mar 14 14:30:37 2023 +0100

        fix gather_sinking_transpose_reshape - check if flatten/unflatten the last dimension

    commit 41e1b72ba9d44a8bd1142735a22feef39e816fcc
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Mar 13 18:26:27 2023 +0100

        fix remove output Gather

    commit 1949dc779a2261686a266bf13fa558078dcce83c
    Merge: 30db0ee2ce b6fe16e051
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Mar 13 18:13:29 2023 +0100

        Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

    commit 30db0ee2ceacf6fb5692421c5647e9ef27794dfb
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Mar 13 18:12:55 2023 +0100

        fix: add GatherSinkingTransposeReshape

    commit b6fe16e051c62cf1b8a1ddf511b2a791587578fc
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 13 17:23:54 2023 +0100

        Reverted C++17 code

    commit fee42e07f7b6861894541cf1d552f82727837a5d
    Merge: 95330967fc 98ead2d3bc
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Mar 13 16:36:36 2023 +0100

        Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

    commit 95330967fc0c39377e33973adeef8c441bb4c905
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Mar 13 16:36:33 2023 +0100

        add ReshapeTransposeSubstitute

    commit 98ead2d3bcc971585d74532ae1263cd7eba8f5b4
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 13 16:33:40 2023 +0100

        Fixed FQ connected to Transpose and Inputs

    commit 6cddb6389e2522c5d9abe262d0a580c01cab4bf8
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 13 16:33:00 2023 +0100

        Added add with convolution fusion

    commit dc705e330f8f23ead78d2e4026eff118028d3dc3
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Fri Mar 10 19:54:19 2023 +0400

        TransposeSinking: add support for Slice op

    commit aba59904c9d95334d6dbea1bdc61971556236d75
    Merge: 3dd79763a4 9844f33807
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Fri Mar 10 15:35:00 2023 +0100

        Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

    commit 9844f33807c0927b813f90a784a77ee7bd725b57
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Thu Mar 9 14:03:12 2023 +0100

        Removed debug

    commit 3dd79763a4ddc31f7a1838070acf341b610c75ea
    Merge: cf3154b4f5 8e010d2cba
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Mar 8 18:27:05 2023 +0100

        Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

    commit 8e010d2cbab5c338ef877e82641b1e813c88bea0
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Wed Mar 8 18:26:04 2023 +0100

        Corrected condition of unsupported permute

    commit cf3154b4f5b0d1f793d1b96e97dec6e440b4fd93
    Merge: 7c0b042e97 95dcc37315
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Mar 8 18:07:37 2023 +0100

        Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

    commit 95dcc3731577db62e42df1c71f2d97068809223e
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Wed Mar 8 18:07:15 2023 +0100

        Removed legacy TransposeWeightsFromNCHWToNHWCPass

    commit 7c0b042e9763a7382b9ba2706be00681a81f49f6
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Mar 8 17:58:02 2023 +0100

        remove debug

    commit 06730afd016185154fbf6751e00da3673b9ce46b
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Wed Mar 8 17:57:57 2023 +0100

        Removed debug code p.2

    commit 37660a9721d103144d88d2429ad0f399d20aef32
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Wed Mar 8 17:51:36 2023 +0100

        Removed  debug code

    commit aa8d2abe3a5e169fe9ebb48fe94845bb4aa1b9f1
    Merge: 02974d7a12 87194dab68
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Wed Mar 8 17:50:28 2023 +0100

        Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

    commit 02974d7a127caf86a4d46701734cb00fa36439ff
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Wed Mar 8 17:50:20 2023 +0100

        Removed legacy parts

    commit 87194dab685aa31fe40624381cecfb40f604a6c8
    Merge: 225e39d499 235a14301d
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Mar 8 17:31:21 2023 +0100

        Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

    commit 225e39d499ed34b7fcd5097c2bfdc86d0d56dc86
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Mar 8 17:31:19 2023 +0100

        fix convert_function_to_cnn_network.cpp not using friendly names; fix transpose_nchw update last node name

    commit 235a14301d1be714e7e0ca074a74ed30120cd327
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Wed Mar 8 17:17:23 2023 +0100

        Fixed validation of gna convolution

    commit dafd9191f52d61b713bb4eec95d63f0a8b0dacfb
    Merge: cf0e43734b 3a63cb5ad8
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Mar 7 16:27:18 2023 +0100

        Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

    commit cf0e43734ba76a7324a1f0975077b798767f6445
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Mar 7 16:26:45 2023 +0100

        fix transpose_nchw friendly names set

    commit 3a63cb5ad8ac449c909d38743650ef5c514d46c4
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Mar 7 15:41:17 2023 +0100

        Extended unsupported transpose condition

    commit 8b3171c2e60bbdb2ada1765d12e67e8039e17005
    Merge: 3385605a9e 69f8479260
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Mar 6 19:53:01 2023 +0100

        Merge branch 'gna_layout_debug' of https://github.com/evkotov/openvino into gna_layout_debug

    commit 3385605a9e08fc32ab0cbd5de702cb736229d43a
    Merge: 8d4fa609a6 d3493c48ae
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Mar 6 19:43:08 2023 +0100

        Merge branch 'gna_convolution_maxpool' into gna_layout_debug

    commit 8d4fa609a67230c89c758dc3aebf22859dcc9498
    Merge: 506128f16c 52b9332306
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Mar 6 19:42:53 2023 +0100

        fix merge

    commit 52b93323062f5d5938af2bf8341a44092380b81b
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Mar 6 19:41:52 2023 +0100

        fix build

    commit 0e2629313690bb97dde765b10d075cba2cf6278a
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Thu Mar 2 14:59:07 2023 +0100

        add binary and gather_fuse

    commit 0ec73f0239e21d25233cd676818f2e873fdf2f81
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Feb 22 13:45:19 2023 +0100

        clang

    commit 28b168c2b4357365bad64983b6f59202726c0a9e
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Feb 22 13:43:36 2023 +0100

        remove debug

    commit fa2a06fb15b5288746deb3b8947b99cdbba4bba7
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Feb 22 13:41:39 2023 +0100

        add descrition

    commit a09bb7f84befe432e7b3563bb213104a69e4f47e
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Feb 22 13:25:54 2023 +0100

        fix transformations restrictions

    commit e723ac91ad3b3821248f2b00008d1ef91678a8dc
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Feb 22 12:23:14 2023 +0100

        refactor

    commit 1cba14e6ed2eb25c62a3987516d3cbdb989b4966
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Feb 22 11:10:31 2023 +0100

        rewrite backward

    commit d127fe60a7af3f3f99c5cca9b4f293e29fa39483
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 21 19:30:51 2023 +0100

        forward sinking - added support 3d

    commit de447274aa81cd594b36842bff7838fd3d0131c8
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 21 17:07:16 2023 +0100

        fix build errors unit tests

    commit b09cf32ccfbc1706c3107273819557adfe3adcef
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 14 14:29:01 2023 +0100

        cleanup tests

    commit 8bb808756cc8d7f0571c35ccd78bda964d37f3c9
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 14 14:16:15 2023 +0100

        wrote unit tests draft

    commit b47571a3cd8a815ac4a84bb140d28304eed9629b
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Feb 13 19:52:32 2023 +0100

        implement transformation for 2d case

    commit 9a5fcd38b0162bbae73f49c482d1c125fdfe3181
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Thu Feb 9 20:40:45 2023 +0100

        added inital transformation

    commit 5e812990b9c8eaafa2b25a55607f8b1fb41f4eff
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Mar 1 19:47:05 2023 +0100

        add unit tests; fix

    commit 01dc4766ead37a455808a7b3b42a849c850144a5
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Mar 1 13:57:34 2023 +0100

        initial

    commit 2c24d97729d93dfc79d26d442e3e4f592ad0614c
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 21 13:36:35 2023 +0100

        fix year

    commit 879f9e3d0ab115ef1a27e709c0aad4a5ce883f30
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 21 13:29:05 2023 +0100

        add docs

    commit 030b2e4fdba826160fa45d16f9ba295c9bebc5ed
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Feb 20 18:58:55 2023 +0100

        cleanup

    commit 97ed5403ea2854064c5817edca11266106b06b95
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Feb 20 18:52:49 2023 +0100

        clang fixes + remove unneeded functions

    commit 4411e6ffbad5f63ce592025785127dda277fe70d
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Feb 20 18:50:03 2023 +0100

        fix unit tests execution

    commit 0483be0d6a31f18dfddfa1bca41ea75a833c0f03
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Feb 20 18:33:03 2023 +0100

        initial

    commit 114314b2a31112bc871b6f0c37568e4f177e9448
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 28 13:52:00 2023 +0100

        initial

    commit d3493c48ae56723dead05d7135d177b9863b3229
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Mar 6 19:38:35 2023 +0100

        fix errors after rebase

    commit 141a77b3934beb912bcfc1aa19d2018cb11dab8f
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 28 13:27:52 2023 +0100

        initial

    commit 69f8479260960c8d9e431ac1d4261264e205fc43
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 6 17:42:13 2023 +0100

        Removed debug switch

    commit a4c43c40ee42c0a025eb9bef7a5ae933245f7c96
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 6 16:47:38 2023 +0100

        Fixed unit tests build

    commit 506128f16c96d2fffe586e6095a98bb87033cf18
    Merge: 4f2b13f34f 451772350c
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 6 16:41:01 2023 +0100

        Merge branch 'feature/extract_gather_transpose'

        Enabled transformations to remove Transpose/Extract layers

    commit 4f2b13f34f8ff723226c82f9f8d13e389e989301
    Merge: 8e5ca732de ba77c7d15b
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 6 16:38:17 2023 +0100

        Merge branch 'tests/bacward_compatibility'

        import/export tests

    commit 8e5ca732dec3b14a87976e8108e587e4acce0da2
    Merge: cab1b42e38 919a7b67bf
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 6 16:35:35 2023 +0100

        Merge branch 'cpu_execution'

        Preprocessing on CPU using ngraph

    commit ba77c7d15b
    Merge: 86e1b68d84 cf8dccaedb
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 6 16:27:46 2023 +0100

        Merge branch 'master' into tests/bacward_compatibility

    commit 451772350c439561dfc4909b22fbafcd47f66011
    Merge: c7cfb95728 cf8dccaedb
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 6 16:22:20 2023 +0100

        Merge branch 'master' of https://github.com/mryzhov/openvino into feature/extract_gather_transpose

    commit cab1b42e38868c5ded27c76c3530917cea7dbae5
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Mar 6 15:39:12 2023 +0100

        fix utils StridedSlice to be sinkable

    commit 849f651f455ae8d75069b058de2f2419c30c9918
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Mar 6 15:24:10 2023 +0100

        Ivan implementation

    commit 9fa205e9e60a8de36edba5b2e4ef05d48c56846a
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Fri Mar 3 18:54:10 2023 +0100

        initial

    commit e40497e9c100e3d0e11b6f03d928a8326fb1626b
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Fri Mar 3 15:18:20 2023 +0100

        add new transformations to pipeline

    commit c39c76bc19a1564816d8d9d1c586b3fdc09a8294
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Thu Mar 2 14:59:07 2023 +0100

        add binary and gather_fuse

    commit ac37d4aa73f5491f9054c0e5228d01c477a980bc
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Thu Mar 2 14:38:43 2023 +0100

        initial

    commit 2d5383d415346d2c6ef460b502e2e53aed6be5f8
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Mar 1 19:47:05 2023 +0100

        add unit tests; fix

    commit 668da966b36d044fb53d29588f3aca3b63907e17
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Mar 1 13:57:34 2023 +0100

        initial

    commit 3e881d1284f5fbf22219b0e2e935a0b72eaec886
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 28 13:52:00 2023 +0100

        initial

    commit c067aec546a4997d7ad84fc4689796aad30d62a9
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Feb 22 13:45:19 2023 +0100

        clang

    commit 4fb2f54fd0b86cc972396958019237f48cf19178
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Feb 22 13:43:36 2023 +0100

        remove debug

    commit 31776582f9e9934bdcfa3cd24381d90719ce6e10
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Feb 22 13:41:39 2023 +0100

        add descrition

    commit 20e390859b9757d1b35c7d3f16b06814a39248ce
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Feb 22 13:25:54 2023 +0100

        fix transformations restrictions

    commit b1cd65d75ff061f9e4ad69ea174fe37e321073d2
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Feb 22 12:23:14 2023 +0100

        refactor

    commit dab693e2b2c859c53ea99364b1721354729185f1
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Feb 22 11:10:31 2023 +0100

        rewrite backward

    commit d33eafcce3ccdf72c46d33216ef1e80645e6a12e
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 21 19:30:51 2023 +0100

        forward sinking - added support 3d

    commit 8676c2a9ce707d370892d1514f1eb8309ee67da6
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 21 17:07:16 2023 +0100

        fix build errors unit tests

    commit 030dab30945018ef6c91879d6c0a034a6e5d3783
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 14 14:29:01 2023 +0100

        cleanup tests

    commit 5dfba630ac973a96b27aa294dc0e090d8fce8b4c
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 14 14:16:15 2023 +0100

        wrote unit tests draft

    commit ae63a5aacc9467de3d74801dd61a26262998f686
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Feb 13 19:52:32 2023 +0100

        implement transformation for 2d case

    commit 23c19be01ea230f8e8ff655e58c6c0b9b8ac70ea
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Thu Feb 9 20:40:45 2023 +0100

        added inital transformation

    commit d69e88c2d2e8354bb3f3e11c24d1ab586eb12401
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 28 13:27:52 2023 +0100

        initial

    commit 82e3f0ed4d0421aec686005a7a4e46de6f2499ac
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 21 13:36:35 2023 +0100

        fix year

    commit b4eb780e588ff12f98277586f5c1da2a22597f55
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Feb 21 13:29:05 2023 +0100

        add docs

    commit fd987b0652856af50463ed5952eec24478bb772b
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Feb 20 18:58:55 2023 +0100

        cleanup

    commit c852d051946312becae279e25bf4b972be6b1556
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Feb 20 18:52:49 2023 +0100

        clang fixes + remove unneeded functions

    commit aa3cc35161104fe568a3301cfe6bd7922f3e5294
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Feb 20 18:50:03 2023 +0100

        fix unit tests execution

    commit c4dd7246e13c154e6e1b3da72768cf794c54df9f
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Feb 20 18:33:03 2023 +0100

        initial

    commit 919a7b67bfb43ff9716fde454cc7fa64308de8af
    Merge: 8df3c54eb8 cf8dccaedb
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 6 16:16:12 2023 +0100

        Merge branch 'master' of https://github.com/mryzhov/openvino into cpu_execution

    commit c7cfb957280e39f3a13c93a0015cd912e594e98e
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 6 16:14:29 2023 +0100

        clang formatted

    commit 6a9b1974992053318d122ddf7b2541069486a36e
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 6 15:53:44 2023 +0100

        clang fixes

    commit 60cbd4d41267c9f2146bb1e13a56c05586e623dd
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 6 15:51:29 2023 +0100

        Excluded gna supported trasposes

    commit b23bbc10a65ebcebcc0acc7298c124c7ddfb5a97
    Merge: 084f147210 716f020956
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 6 11:47:34 2023 +0100

        Merge branch 'feature/extract_gather_transpose' of https://github.com/mryzhov/openvino into feature/extract_gather_transpose

    commit 084f1472104e8a767483646856a54b50af4af408
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Mar 6 11:47:29 2023 +0100

        Fixed double transposition

    commit 716f020956fe249672b0aea6c3bd30a081cfca6d
    Merge: 86ea8cb7dc 07f287e362
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 28 16:55:55 2023 +0100

        Merge branch 'master' into feature/extract_gather_transpose

    commit 86ea8cb7dc2e88ffb40cf17dfa8de28af4144814
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 28 16:12:15 2023 +0100

        clang fixes

    commit 38e69687943634430b77ebf2059771c1b13f5d96
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 28 12:23:07 2023 +0100

        Added tests

    commit b5bda65d1b67c9d5a3d2a81cc6c8a0e3e108bc87
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 28 12:22:47 2023 +0100

        fixed compile error

    commit 7815caa8e45d7498d9c56bcb2443a6ef6762da86
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 28 11:06:39 2023 +0100

        Init

    commit 4e4da5b0097ecb055e1faf118931a148842a07e4
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Thu Feb 23 10:24:57 2023 +0100

        Gather transformation

    commit 8df3c54eb8a132f99e7a579269db57478ff738e3
    Merge: e0e80f9dd8 98392a043b
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Wed Feb 22 17:04:55 2023 +0100

        Merge branch 'master' into cpu_execution

    commit e0e80f9dd886204768a129eb426b1c6ec1a114c3
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Wed Feb 22 14:56:49 2023 +0100

        WA I16->F16

    commit 1d6319c257bd7d6d99e952286b86c3d8e05bcf00
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 21 15:50:29 2023 +0100

        Reverted test

    commit 86e1b68d84
    Merge: 9871d61042 94b64fed79
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 21 12:28:11 2023 +0100

        Merge branch 'master' into tests/bacward_compatibility

    commit cd2c75cbd35975a8b3c93b8001dc38f6fab38f46
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 21 12:23:52 2023 +0100

        fixed static build

    commit 290b592f4ace79d3bbff2906186481bfd5e267a0
    Merge: d63cfcf774 94b64fed79
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 21 12:01:38 2023 +0100

        Merge branch 'master' into cpu_execution

    commit d63cfcf7745652cfdfd38211ae3565153860a018
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 21 10:55:52 2023 +0100

        clang format

    commit 71fedeef5d9c12b6bfe95e8ec3d2d6686d1619c6
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 21 10:30:25 2023 +0100

         Revert "backward tests"

        This reverts commit 05dacd1a51a8756880e49523138af9b14245e262.

    commit 036fb95e79307c90e2f07add88683f47e5d0a857
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Feb 20 16:07:52 2023 +0100

        backward tests

    commit ed6b8d5b0da6f4dde4489681775cc5a26af0bc13
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Feb 20 11:38:38 2023 +0100

        reset gather_transformation.cpp to master

    commit 64c7f4e963e439151252d7d339baadd7d2722581
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Feb 20 11:36:26 2023 +0100

        reverted

    commit 9af3771cc31d61aa1eb09053c56625aa7e7c5927
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Feb 20 11:31:59 2023 +0100

        removed changes in gather

    commit d0a23572b5fad9176e58b816b0ab2ca0697119ff
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Feb 20 11:28:48 2023 +0100

        Revert "Added backward compatibility tests"

        This reverts commit afdd7afe72df88c96593f9f5b9bbfaaf222cec63.

    commit a504e700fcda8ada2a4753b71dd25b517951a47a
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Feb 20 11:25:00 2023 +0100

        Added trasposition check

    commit 560df30526beb7e5da57f8f2c42dbc4e63029da4
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Thu Feb 16 18:32:25 2023 +0100

        Removed Gather transformation

    commit 2c5b99d224df8acf39330483065e828de46af03b
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Thu Feb 16 15:30:32 2023 +0100

        Added backward compatibility tests

    commit e4935d02f5850b2f526412b0c30ee650f8175343
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Feb 13 21:14:29 2023 +0100

        compile warnings

    commit 86a36411027aa7aba97784c261ffcb64dd2ff92f
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Feb 13 18:14:21 2023 +0100

        clang formatting

    commit 308b239302c6ce0186cb95bfc31ccfd5122b1613
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Mon Feb 13 17:24:51 2023 +0100

        Fixed merge issues

    commit 94818a4de96c068241b285858bb9317d20c13c94
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Thu Feb 9 16:39:47 2023 +0100

        passed all the tests

    commit fa5e3740e2d3f961ade7b6b894d11d23c549a4d6
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 7 13:02:21 2023 +0100

        Fixed precison in gather transformation

    commit f0a70f054b251999d36b3291cd90a2c9e63e5f17
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 7 13:01:43 2023 +0100

        Added the new export version

    commit cef818c7a2e0a22fdfd215576f60975070debebd
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 7 12:58:00 2023 +0100

        Support of legacy transposes

    commit e71e476dd5c2feb2f699987d1dbade1271a45e88
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Thu Jan 26 16:47:41 2023 +0100

        Pre_post_process init

    commit e1d297092c952fa9ee218b056632964a8941d2ec
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Thu Jan 19 14:52:02 2023 +0100

        fix after rebase

    commit 45ab28de3aa6d6de42c9378a2952a1dbc23d3ba9
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Nov 23 19:10:24 2022 +0100

        remove GatherIESubstitute transformation as not needed

    commit 4cb2b0aa3301ee68a0204cf3723a6465951626e1
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Wed Nov 23 19:08:57 2022 +0100

        fix funcitional test

    commit 981d086f7f327f7b6665b6076da4d5aaff36be2e
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Tue Nov 22 14:23:47 2022 +0100

        fix precision; left bug output EXACT

    commit 3cf71e0de1a47ac771d5611014d8e7e109168043
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Fri Nov 18 18:45:12 2022 +0100

        fixes

    commit c69d57f1f3b52dfbf6c68279353e4ca95eb26648
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Fri Nov 18 17:55:35 2022 +0100

        fix functional test

    commit 1b540338910e2ad17f21ba7d71c098877dadfa03
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Nov 14 17:51:20 2022 +0100

        add gather remove

    commit d7bff1ffa868b41f340cd17e66548d630b0822c2
    Author: Evgeny Kotov <evgeny.kotov@intel.com>
    Date:   Mon Nov 14 16:11:44 2022 +0100

        initial

    commit 9871d61042
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 21 11:10:38 2023 +0100

        Fixed build of deprecated tests

    commit cbf5246c1a
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 21 10:46:15 2023 +0100

        clang format

    commit c34f81b899
    Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
    Date:   Tue Feb 21 10:28:33 2023 +0100

        Migrated the deprecated backward compatibility tests to the new infra

commit bc1dd9d302
Author: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Date:   Sun Mar 19 18:15:21 2023 +0100

    Squashed commit of the following:

    commit a56a0768f1
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Fri Mar 17 15:48:23 2023 +0000

        remove TransposeSinking from MOC

    commit 430bdbec35
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Fri Mar 17 15:39:22 2023 +0000

        delete debug serialize

    commit a40658e369
    Merge: 346796af9c cf70ced2a9
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Fri Mar 17 15:33:46 2023 +0000

        Merge branch 'itikhono/ts/slice' of https://github.com/itikhono/openvino into itikhono/ts/slice

    commit 346796af9c
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Fri Mar 17 15:33:27 2023 +0000

        fix TSSqueeze/TSUnsqueeze transformations

    commit cf70ced2a9
    Merge: 7a6988a4a6 a9bd5f741d
    Author: Ivan Tikhonov <ivan.tikhonov@intel.com>
    Date:   Fri Mar 17 13:12:18 2023 +0400

        Merge branch 'master' into itikhono/ts/slice

    commit 7a6988a4a6
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Fri Mar 17 08:50:15 2023 +0000

        codestyle

    commit 849dc70763
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Fri Mar 17 16:32:45 2023 +0400

        refactoring

    commit 958f000e02
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Fri Mar 17 16:20:06 2023 +0400

        Added TSSlice transformation to TSGeneral, created TransposeSinkingGeneral alias in ov::pass namespace

    commit 83ab2cc5f6
    Merge: 1f8fdfb353 88ddbb2437
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Fri Mar 17 16:09:35 2023 +0400

        Merge branch 'itikhono/ts/slice' of https://github.com/itikhono/openvino into itikhono/ts/slice

    commit 1f8fdfb353
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Fri Mar 17 16:06:15 2023 +0400

        fix warnings

    commit 66d16ae45e
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Fri Mar 17 00:07:44 2023 +0400

        fix minor mistakes

    commit 1ac7690d42
    Merge: f1dc3702f1 6ffa8da922
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Fri Mar 17 00:02:39 2023 +0400

        Merge remote-tracking branch 'upstream/master' into itikhono/ts/slice

    commit f1dc3702f1
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Mar 16 23:48:10 2023 +0400

        TSReduction refactoring, move Unsqueeze/Squeeze transformations to separate files, added limited support for Reshape op + tests

    commit 88ddbb2437
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Mar 16 16:16:59 2023 +0000

        codestyle

    commit 2bc1334f65
    Merge: ce84ce24ec 4b9e048e18
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Wed Mar 15 17:10:30 2023 +0400

        Merge branch 'itikhono/ts/refactoring' into itikhono/ts/slice

    commit ce84ce24ec
    Merge: 6e6361f83a 36c18e29a8
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Wed Mar 15 16:25:41 2023 +0400

        Merge remote-tracking branch 'upstream/master' into itikhono/ts/slice

    commit 4b9e048e18
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Mar 14 22:54:33 2023 +0400

        resolve merge conflicts

    commit 169a722212
    Merge: ee42aaf13b 596036a2db
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Mar 14 22:48:22 2023 +0400

        Merge remote-tracking branch 'upstream/master' into itikhono/ts/refactoring

    commit ee42aaf13b
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Mar 14 14:43:50 2023 +0000

        codestyle

    commit b3342f7c46
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Mar 14 22:30:09 2023 +0400

        TransposeSinking refactoring: class names, namespaces

    commit e7759e1fa0
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Mar 14 21:59:28 2023 +0400

        TransposeSinking Rafactoring part2: move the transformations to a separate folder, align namespaces

    commit 7b91ac1e6b
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Mar 14 17:59:12 2023 +0400

        fix a warning

    commit d284ac1b7a
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Mar 14 17:48:46 2023 +0400

        Add descriptions to the transformations, add additional checks

    commit 6e6361f83a
    Merge: 54bf0444e4 3a96e06d4c
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Mar 14 16:20:34 2023 +0400

        Merge remote-tracking branch 'upstream/master' into itikhono/ts/slice

    commit 54bf0444e4
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Fri Mar 10 19:54:19 2023 +0400

        TransposeSinking: add support for Slice op

    commit a47a18cf55
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Mar 9 22:29:37 2023 +0400

        Add new tests for Unsqueeze/Squeeze; refactoring; remove debug code

    commit e13d0e589f
    Merge: fa8e81f0f1 3d52fc843a
    Author: Ivan Tikhonov <ivan.tikhonov@intel.com>
    Date:   Thu Mar 9 01:43:47 2023 +0400

        Merge branch 'master' into itikhono/ts/fix_performance_issues

    commit fa8e81f0f1
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Mar 9 04:20:08 2023 +0400

        Fix warning as error on Windows

    commit c77e98caf6
    Merge: 99192feb02 3eeaf7f9bd
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Mar 9 02:42:47 2023 +0400

        Merge branch 'itikhono/ts/fix_performance_issues' of https://github.com/itikhono/openvino into itikhono/ts/fix_performance_issues

    commit 99192feb02
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Mar 9 02:42:37 2023 +0400

        fix unary tests

    commit 11b500953d
    Merge: b60015f90d 3b924ca206
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Mar 9 01:33:57 2023 +0400

        Merge branch 'itikhono/ts/fix_performance_issues' of https://github.com/itikhono/openvino into itikhono/ts/fix_performance_issues

    commit b60015f90d
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Mar 9 01:33:46 2023 +0400

        Add tests for TS backward transformations, update TransposeSinkingFuse transformation, delete StridedSlice transformation prototype + tests refactoring

    commit 3eeaf7f9bd
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Wed Mar 8 17:42:25 2023 +0000

        codestyle

    commit db09fe4965
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Mar 7 16:55:40 2023 +0400

        fix build

    commit 3b924ca206
    Merge: 54b4061282 db09fe4965
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Mar 7 08:56:32 2023 +0000

        Merge branch 'itikhono/ts/fix_performance_issues' of https://github.com/itikhono/openvino into itikhono/ts/fix_performance_issues

    commit 54b4061282
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Mar 7 08:51:48 2023 +0000

        TransposeReduction: skip the case when 2nd input for Squeeze is not provided

    commit 176686318f
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Mar 7 07:34:40 2023 +0000

        codestye

    commit 3c5f62c013
    Merge: ef0e89551d 0dad7749b5
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Mar 7 07:33:09 2023 +0000

        Merge remote-tracking branch 'upstream/master' into itikhono/ts/fix_performance_issues

    commit ef0e89551d
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Mar 7 15:30:22 2023 +0400

        Transpose sinking tests refactoring: part 3. + Revert changes in MOC.

    commit d71949fd09
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Mon Mar 6 18:23:40 2023 +0400

        Add limited support for StridedSlice op

    commit 3565ff2181
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Mon Mar 6 17:45:52 2023 +0400

        TransposeSinking tests refactoring: part2

    commit 34c89eb962
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Fri Mar 3 04:34:48 2023 +0400

        TransposeSinking tests refactoring: part1

    commit c5991f0b06
    Merge: 6de1336894 caea77c132
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Mar 2 17:21:17 2023 +0000

        Merge branch 'itikhono/ts/fix_performance_issues' of https://github.com/itikhono/openvino into itikhono/ts/fix_performance_issues

    commit caea77c132
    Merge: 199114a4c6 873d15ce1a
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Mar 2 21:39:57 2023 +0400

        Merge branch 'itikhono/ts/fix_performance_issues' of https://github.com/itikhono/openvino into itikhono/ts/fix_performance_issues

    commit 199114a4c6
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Mar 2 21:39:12 2023 +0400

        add include

    commit 6de1336894
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Mar 2 17:21:02 2023 +0000

        add the missed line

    commit 123835c86d
    Author: Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Mar 2 19:09:36 2023 +0400

        Support TS for Interpolate, VariadicSplit, IsInf, IsNan, IsFinite + refactoring

    commit 873d15ce1a
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Mar 2 13:33:40 2023 +0000

        fix gna build

    commit 13f17d254b
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Mar 2 12:24:12 2023 +0000

        fix TS for Interpolate + codestyle

    commit b769d21912
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Feb 23 17:14:30 2023 +0000

        Add SpaceToBatch/BatchToSpace

    commit 9d8016d1e6
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Feb 23 17:12:54 2023 +0000

        Fix validate for split, revert changes for concat, add BatchToSpace/SpaceToBatch

    commit 20579455b7
    Merge: 3baf0c7900 d9fc5bac80
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Feb 23 10:32:14 2023 +0000

        Merge remote-tracking branch 'upstream/master' into itikhono/ts/fix_performance_issues

    commit 3baf0c7900
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Thu Feb 23 10:31:56 2023 +0000

        fix node validation

    commit fa9fe34c16
    Merge: ef6e141082 eaf368a5f5
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Wed Feb 22 15:33:52 2023 +0000

        Merge remote-tracking branch 'upstream/master' into itikhono/ts/fix_performance_issues

    commit ef6e141082
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Feb 21 14:34:26 2023 +0000

        Add additional validations

    commit 81b9e6eece
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Mon Feb 20 13:54:27 2023 +0000

        delete debug print

    commit 9333c1cac5
    Merge: aa1aadd3c4 08293d39e1
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Mon Feb 20 13:51:45 2023 +0000

        Merge branch 'itikhono/ts/fix_performance_issues' of https://github.com/itikhono/openvino into itikhono/ts/fix_performance_issues

    commit aa1aadd3c4
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Mon Feb 20 13:50:42 2023 +0000

        Fix TransposeReduction, fix TransposeSinkingSplit, add unsqueeze support

    commit 20168b251a
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Fri Feb 17 07:46:12 2023 +0000

        fix TransposeReduce transformations

    commit 08293d39e1
    Merge: 43ef82320f 5cc1c2c5e5
    Author: Ivan Tikhonov <ivan.tikhonov@intel.com>
    Date:   Tue Feb 14 18:11:20 2023 +0400

        Merge branch 'master' into itikhono/ts/fix_performance_issues

    commit 43ef82320f
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Feb 14 14:09:25 2023 +0000

        fix the issue in TransposeFuse transformation

    commit 6e3fcf74e2
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Tue Feb 14 08:09:25 2023 +0000

        Fix TransposeReduceBackward

    commit a34e3ced9e
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Mon Feb 13 17:08:19 2023 +0000

        fix ts for Concat and Reduce

    commit 8598d64174
    Merge: d9ea97bf4b f5bff5c087
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Mon Feb 13 07:46:32 2023 +0000

        Merge branch 'itikhono/ts/fix_performance_issues' of https://github.com/itikhono/openvino into itikhono/ts/fix_performance_issues

    commit d9ea97bf4b
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Mon Feb 13 07:45:54 2023 +0000

        fix warning as error, fix tests failures

    commit f5bff5c087
    Merge: 1de806f9f7 b300df1be6
    Author: Ivan Tikhonov <ivan.tikhonov@intel.com>
    Date:   Sun Feb 12 21:51:00 2023 +0400

        Merge branch 'master' into itikhono/ts/fix_performance_issues

    commit 1de806f9f7
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Sun Feb 12 17:48:05 2023 +0000

        codestyle

    commit 2c48d6c583
    Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
    Date:   Sun Feb 12 17:47:32 2023 +0000

        Resolve the performance issues in TransposeSinking transformation

commit db220f11ee
Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
Date:   Tue Mar 21 09:42:42 2023 +0000

    delete debug code

commit 8f8f0e821b
Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
Date:   Tue Mar 21 09:41:48 2023 +0000

    fix TSSqueeze/TSUnsqueeze transformations in case of Reshape op

commit 981e8ad3c0
Merge: a56a0768f1 c5f65eea73
Author: Ivan <ivan.tikhonov@intel.com>
Date:   Mon Mar 20 19:47:18 2023 +0400

    Merge remote-tracking branch 'upstream/master' into itikhono/ts/slice

commit a56a0768f1
Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
Date:   Fri Mar 17 15:48:23 2023 +0000

    remove TransposeSinking from MOC

commit 430bdbec35
Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
Date:   Fri Mar 17 15:39:22 2023 +0000

    delete debug serialize

commit a40658e369
Merge: 346796af9c cf70ced2a9
Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
Date:   Fri Mar 17 15:33:46 2023 +0000

    Merge branch 'itikhono/ts/slice' of https://github.com/itikhono/openvino into itikhono/ts/slice

commit 346796af9c
Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
Date:   Fri Mar 17 15:33:27 2023 +0000

    fix TSSqueeze/TSUnsqueeze transformations

commit cf70ced2a9
Merge: 7a6988a4a6 a9bd5f741d
Author: Ivan Tikhonov <ivan.tikhonov@intel.com>
Date:   Fri Mar 17 13:12:18 2023 +0400

    Merge branch 'master' into itikhono/ts/slice

commit 7a6988a4a6
Author: Tikhonov Ivan <ivan.tikhonov@intel.com>
Date:   Fri Mar 17 08:50:15 2023 +0000

    codestyle

commit 849dc70763
Author: Ivan <ivan.tikhonov@intel.com>
Date:   Fri Mar 17 16:32:45 2023 +0400

    refactoring

commit 958f000e02
Author: Ivan <ivan.tikhonov@intel.com>
Date:   Fri Mar 17 16:20:06 2023 +0400

    Added TSSlice transformation to TSG…

* Merge fixes

* clang fixes

* fix Reshape is unsupported; Transpose backward sinking case

* fix functional test params

* fix bug : duplicated source code in GNAGraphCompiler::PoolingPrimitive after rebase

* remove legacy transformations

* Fixed out of range issue

* aligned memory for convolution with padding

# Conflicts:
#	src/plugins/intel_gna/tests/functional/shared_tests_instances/subgraph_tests/const_conv_concat.cpp

* clang fixes

* [GNA] Limitations refactoring (#16957)

* Limitations refactoring

* fix CI builds/tests

* changes after review

* Move GraphCompiler initialization to constructor

* resolve conflicts after rebase

* update after review

* resolve problem with double initialization for Limitations
# Conflicts:
#	src/plugins/intel_gna/src/backend/gna_limitations.cpp
#	src/plugins/intel_gna/src/backend/gna_limitations.hpp
#	src/plugins/intel_gna/src/gna_transformations_pipeline.cpp

* Rebase fixes

* Removed duplicate of is_transpose_supported

* Rebase fix 2

* Reduced code duplication

* Clang code style

* fix unit tests

* Removed legacy failed tests

* add gather_sinking_test

* Corrected  tests to produce unsupported transposes

* Reverted unnecessary fix

* hide deprecated warnings

* clang fix

* fixed concat-gather transformation

* revert back ts_squeeze bugfix

* remove workaround

* fix build

* remove concat_transpose.cpp

* remove backward_compatibility.cpp test

* remove debug

* fix std::equal

* Revert "remove debug"

This reverts commit 574a3a04f3.

* remove debug

* return concat_transpose with fix

* rotates convolution inputs

* fix after rebase

* cleanup after rebase

* fixes after rebase

* fixes after rebase

* fix after rebase GatherSinkingGeneralBackward - return back GatherSinkingSplitBackward

* fix after rebase GatherSinkingGeneralBackward - return back

* clang fix

* windows build fix; fix rebase issue

* code review fixes

* clang fixes

* code review fixes

* code review fixes

* code review fixes - rename ts_split and ts_concat files

* code review fixes : move auxilary functions from pipeline to transformation helper

* code review fixes -> move has_one_consumer to graph_utils

* fix comparing

* cleanup

* code review fix - graph_compiler remove unneeded copying

* code review fixes - remove unneeded code

* code review fix gna_layer_info

* code review fix - remove unneeded code from LoadNetwork

* cleanup

* update modules

* code review fixes

* code review fixes

* code review fixes

* Revert "code review fixes -> move has_one_consumer to graph_utils"

This reverts commit f1a24041083b0da3c5ca34417e1789fd54e24dab.

* Revert "code review fixes"

This reverts commit d01b58a5b1a7a70bf16eec6f86b8dc1e50bf65c7.

* revert commits

* code review fixes

* code review fixes - remove unneeded

* code review fixes

* code review fix

---------

Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Co-authored-by: Tomasz Adamowicz <tomasz.adamowicz@intel.com>
This commit is contained in:
Evgeny Kotov
2023-06-28 19:55:34 +02:00
committed by GitHub
parent 18e737493c
commit 7ae8826b52
42 changed files with 1149 additions and 542 deletions

View File

@@ -18,6 +18,7 @@
#include "gna_lib_ver_selector.hpp"
#include "ie_ngraph_utils.hpp"
#include "log/log.hpp"
#include "openvino/opsets/opset12.hpp"
namespace std {
inline std::ostream& operator<<(std::ostream& os, const std::set<ov::element::Type>& t) {
@@ -35,6 +36,7 @@ inline std::ostream& operator<<(std::ostream& os, const std::set<ov::element::Ty
namespace ov {
namespace intel_gna {
using namespace target;
using namespace opset12;
namespace limitations {
class SupportedElementTypes {
@@ -689,22 +691,6 @@ void Limitations::init(const DeviceVersion& compile_target) {
k_instance = std::shared_ptr<Limitations>(new Limitations(compile_target));
}
bool Limitations::is_transpose_2d(const std::vector<size_t>& shape) {
return std::count_if(std::begin(shape), std::end(shape), [](size_t dim) {
return dim != 1;
}) == 2;
}
bool Limitations::is_transpose_supported(const std::vector<size_t>& shape) {
if (!is_transpose_2d(shape))
return false;
auto shape_no_1 = shape;
shape_no_1.erase(std::remove(shape_no_1.begin(), shape_no_1.end(), 1), shape_no_1.end());
size_t min, max;
std::tie(min, max) = std::minmax(shape_no_1[0], shape_no_1[1]);
return min <= 8 && max % 8 == 0 && max >= 8 && max <= kTransposeMaxSize;
}
size_t Limitations::get_min_batch_to_fit_in_buffer(InferenceEngine::DataPtr input) {
auto total_size = InferenceEngine::details::product(std::begin(input->getDims()), std::end(input->getDims()));
return total_size / kBufferMaxSize + 1;
@@ -753,31 +739,38 @@ bool SupportedElementTypes::IsConstantTypeSupported(ov::element::Type elem_type,
return true;
}
bool Limitations::is_transpose_supported(const std::shared_ptr<const ov::Node>& node) {
OPENVINO_ASSERT(node, "Transpose node is empty!");
const ov::Shape squeezed_shape = graph_utils::squeeze_shape(node->get_input_shape(0));
const size_t min_input_dim = std::min(squeezed_shape[0], squeezed_shape[1]);
const size_t max_input_dim = std::max(squeezed_shape[0], squeezed_shape[1]);
bool Limitations::is_transpose_supported(const ov::Shape& shape) {
const ov::Shape squeezed_shape = graph_utils::squeeze_shape(shape);
// GNA transpose limitations:
// - supports 2d transposes only
// - smaller dimension should be less or equal to 8
// - bigger dimension should be a multiple of Limitations::kNoOfInputsDivisor
if (squeezed_shape.size() == 2 && min_input_dim <= 8 && ALIGN(max_input_dim, kNoOfInputsDivisor) == max_input_dim) {
return true;
// - bigger dimension should be a multiple of limitations::noOfInputsDivisor
if (squeezed_shape.size() == 2) {
const size_t min_input_dim = std::min(squeezed_shape[0], squeezed_shape[1]);
const size_t max_input_dim = std::max(squeezed_shape[0], squeezed_shape[1]);
if (min_input_dim <= 8 && max_input_dim % Limitations::kNoOfInputsDivisor == 0 &&
max_input_dim <= kTransposeMaxSize) {
return true;
}
}
return false;
}
bool Limitations::is_conv_supported(const std::shared_ptr<ngraph::op::ConvolutionIE>& conv_ie,
bool Limitations::is_transpose_supported(const std::shared_ptr<const ov::Node>& node) {
OPENVINO_ASSERT(node, "Transpose node is empty!");
return is_transpose_supported(node->get_input_shape(0));
}
bool Limitations::is_conv_supported(const std::shared_ptr<ov::intel_gna::op::GNAConvolution>& conv_gna,
const InferenceEngine::Precision gna_precision,
bool is_exception_allowed) {
OPENVINO_ASSERT(conv_ie, "ConvolutionIE node is empty!");
size_t batch_size = conv_ie->input_value(0).get_shape()[0];
OPENVINO_ASSERT(conv_gna, "GNAConvolution node is empty!");
size_t batch_size = conv_gna->input_value(0).get_shape()[0];
if (batch_size != 1) {
if (is_exception_allowed) {
THROW_GNA_EXCEPTION << "topology with layer: " + conv_ie->get_friendly_name() +
", type: " + conv_ie->get_type_name() + ", and batch size(" +
THROW_GNA_EXCEPTION << "topology with layer: " + conv_gna->get_friendly_name() +
", type: " + conv_gna->get_type_name() + ", and batch size(" +
std::to_string(batch_size) + ") != 1 not supported";
}
return false;
@@ -789,15 +782,15 @@ bool Limitations::is_conv_supported(const std::shared_ptr<ngraph::op::Convolutio
static_cast<uint32_t>(filter_stride_width));
return cnn2d::AbstractValidator::ValidationSuccesful(is_exception_allowed,
error,
conv_ie->get_friendly_name(),
conv_ie->get_type_name());
conv_gna->get_friendly_name(),
conv_gna->get_type_name());
};
auto input_shape = conv_ie->input_value(0).get_shape();
auto filter_shape = conv_ie->input_value(1).get_shape();
if ((4 == filter_shape.size() && filter_shape[2] > 1 && filter_shape[3] > 1) ||
(4 == input_shape.size() && input_shape[2] > 1 && input_shape[3] > 1)) {
auto input_shape = conv_gna->input_value(0).get_shape();
auto filter_shape = conv_gna->input_value(1).get_shape();
if ((4 == filter_shape.size() && filter_shape[1] > 1 && filter_shape[2] > 1) ||
(4 == input_shape.size() && input_shape[1] > 1 && input_shape[2] > 1)) {
pass::helper::ConvData conv_data;
pass::helper::GetConvData(conv_ie, conv_data);
pass::helper::GetConvData(conv_gna, conv_data);
if (gna_convolution_layer::isMappableFrom2DTo1D(static_cast<uint32_t>(conv_data.input_height),
static_cast<uint32_t>(conv_data.input_width),
static_cast<uint32_t>(conv_data.input_channel_count),
@@ -809,7 +802,7 @@ bool Limitations::is_conv_supported(const std::shared_ptr<ngraph::op::Convolutio
}
if (m_cnn_validator) {
return m_cnn_validator->ValidateCnn2D(conv_ie->get_friendly_name(),
return m_cnn_validator->ValidateCnn2D(conv_gna->get_friendly_name(),
static_cast<uint32_t>(conv_data.input_height),
static_cast<uint32_t>(conv_data.input_width),
static_cast<uint32_t>(conv_data.input_channel_count),
@@ -824,10 +817,12 @@ bool Limitations::is_conv_supported(const std::shared_ptr<ngraph::op::Convolutio
is_exception_allowed);
}
}
return check_dilation(conv_ie->get_dilations()[0], conv_ie->get_dilations()[1]);
return check_dilation(conv_gna->get_dilations()[0],
conv_gna->get_dilations()[conv_gna->get_dilations().size() - 1]);
}
bool Limitations::is_pooling_supported(const std::shared_ptr<ngraph::opset7::MaxPool> max_pool,
bool Limitations::is_pooling_supported(const std::shared_ptr<ov::intel_gna::op::GNAMaxPool> max_pool,
bool is_exception_allowed) {
OPENVINO_ASSERT(max_pool, "MaxPool node is empty!");
auto kernels = max_pool->get_kernel();
@@ -869,6 +864,100 @@ bool Limitations::is_split_supported(const std::shared_ptr<ov::Node>& node, bool
return is_aligned;
}
bool Limitations::is_concat_supported(const std::shared_ptr<const ov::Node>& node) {
OPENVINO_ASSERT(node, "Concat node is empty!");
auto concat_node = std::dynamic_pointer_cast<const Concat>(node);
const ov::Shape& output_shape = concat_node->get_output_shape(0);
auto axis = concat_node->get_axis();
return graph_utils::get_first_valuable_dim_id(output_shape) == axis;
}
bool Limitations::is_forward_transposed_concat_supported(const std::shared_ptr<const ov::Node>& node,
const AxisVector& order) {
auto concat_node = std::dynamic_pointer_cast<const Concat>(node);
if (!concat_node) {
log::debug() << "Concat node is empty!" << std::endl;
return false;
}
const ov::Shape& output_shape = concat_node->get_output_shape(0);
auto axis = concat_node->get_axis();
const ov::Shape& transposed_shape =
graph_utils::transpose_shape(output_shape, pass::helper::reverse_transpose_order(order));
const size_t transposed_concat_axis = order[axis];
return graph_utils::get_first_valuable_dim_id(transposed_shape) == static_cast<int64_t>(transposed_concat_axis);
}
bool Limitations::is_backward_transposed_concat_supported(const std::shared_ptr<const ov::Node>& node,
const AxisVector& order) {
auto concat_node = std::dynamic_pointer_cast<const Concat>(node);
if (!concat_node) {
log::debug() << "Concat node is empty!" << std::endl;
return false;
}
const ov::Shape& output_shape = concat_node->get_output_shape(0);
auto axis = concat_node->get_axis();
const ov::Shape& transposed_shape = graph_utils::transpose_shape(output_shape, order);
const size_t transposed_concat_axis = order[axis];
return graph_utils::get_first_valuable_dim_id(transposed_shape) == static_cast<int64_t>(transposed_concat_axis);
}
bool Limitations::is_forward_transposed_split_supported(const std::shared_ptr<const ov::Node>& node,
const AxisVector& order) {
std::shared_ptr<const ov::Node> split_node = nullptr;
if (std::dynamic_pointer_cast<const Split>(node)) {
split_node = std::dynamic_pointer_cast<const Split>(node);
} else if (std::dynamic_pointer_cast<const VariadicSplit>(node)) {
split_node = std::dynamic_pointer_cast<const VariadicSplit>(node);
} else {
log::debug() << "Split node is empty!" << std::endl;
return false;
}
const ov::Shape& output_shape = split_node->get_output_shape(0);
auto constant_node = as_type_ptr<Constant>(split_node->input_value(1).get_node_shared_ptr());
if (!constant_node)
return false;
auto axis = constant_node->get_axis_vector_val()[0];
const ov::Shape& transposed_shape =
graph_utils::transpose_shape(output_shape, pass::helper::reverse_transpose_order(order));
const size_t transposed_concat_axis = order[axis];
return graph_utils::get_first_valuable_dim_id(transposed_shape) == static_cast<int64_t>(transposed_concat_axis);
}
bool Limitations::is_backward_transposed_split_supported(const std::shared_ptr<const ov::Node>& node,
const AxisVector& order) {
std::shared_ptr<const ov::Node> split_node = nullptr;
if (std::dynamic_pointer_cast<const Split>(node)) {
split_node = std::dynamic_pointer_cast<const Split>(node);
} else if (std::dynamic_pointer_cast<const VariadicSplit>(node)) {
split_node = std::dynamic_pointer_cast<const VariadicSplit>(node);
} else {
log::debug() << "Split node is empty!" << std::endl;
return false;
}
const ov::Shape& output_shape = split_node->get_output_shape(0);
auto constant_node = as_type_ptr<Constant>(split_node->input_value(1).get_node_shared_ptr());
if (!constant_node)
return false;
auto axis = constant_node->get_axis_vector_val()[0];
const ov::Shape& transposed_shape =
graph_utils::transpose_shape(output_shape, pass::helper::reverse_transpose_order(order));
const int64_t transposed_concat_axis = order[axis];
return graph_utils::get_first_valuable_dim_id(transposed_shape) == transposed_concat_axis;
}
bool Limitations::is_op_supported(const std::shared_ptr<ov::Node>& node,
const InferenceEngine::Precision gna_precision,
bool is_exception_allowed) {
@@ -876,12 +965,13 @@ bool Limitations::is_op_supported(const std::shared_ptr<ov::Node>& node,
return SupportedElementTypes::IsParameterTypeSupported(node->get_element_type(), is_exception_allowed);
} else if (ov::op::util::is_constant(node)) {
return SupportedElementTypes::IsConstantTypeSupported(node->get_element_type(), is_exception_allowed);
} else if (auto conv_ie = std::dynamic_pointer_cast<ngraph::op::ConvolutionIE>(node)) {
return is_conv_supported(conv_ie, gna_precision, is_exception_allowed);
} else if (auto conv = std::dynamic_pointer_cast<ov::intel_gna::op::GNAConvolution>(node)) {
return is_conv_supported(conv, gna_precision, is_exception_allowed);
} else if (auto fully_connected = std::dynamic_pointer_cast<ngraph::op::FullyConnected>(node)) {
return is_fc_supported(fully_connected, is_exception_allowed);
} else if (ov::intel_gna::graph_utils::is_pooling(node)) {
return is_pooling_supported(std::dynamic_pointer_cast<ngraph::opset7::MaxPool>(node), is_exception_allowed);
return is_pooling_supported(std::dynamic_pointer_cast<ov::intel_gna::op::GNAMaxPool>(node),
is_exception_allowed);
} else if (ov::op::util::is_output(node) || ov::op::util::is_sink(node) ||
ov::intel_gna::graph_utils::is_eltwise_add(node) || ov::intel_gna::graph_utils::is_eltwise_mul(node) ||
ov::intel_gna::graph_utils::is_crop_affined(node) ||
@@ -891,11 +981,11 @@ bool Limitations::is_op_supported(const std::shared_ptr<ov::Node>& node,
(std::dynamic_pointer_cast<ov::op::util::ReadValueBase>(node) != nullptr) ||
(std::dynamic_pointer_cast<ngraph::op::ScaleShiftIE>(node) != nullptr) ||
(std::dynamic_pointer_cast<ngraph::op::PowerIE>(node) != nullptr) ||
(std::dynamic_pointer_cast<ngraph::opset9::MatMul>(node) != nullptr)) {
(std::dynamic_pointer_cast<MatMul>(node) != nullptr)) {
return true;
} else if (ov::intel_gna::graph_utils::is_gna_precision_agnostic(node)) {
if ((std::dynamic_pointer_cast<ngraph::opset9::Split>(node) != nullptr) ||
(std::dynamic_pointer_cast<ngraph::opset9::VariadicSplit>(node) != nullptr)) {
if ((std::dynamic_pointer_cast<Split>(node) != nullptr) ||
(std::dynamic_pointer_cast<VariadicSplit>(node) != nullptr)) {
return is_split_supported(node, is_exception_allowed);
}
// TODO check concat are aligned when transformation will be moved to ngraph

View File

@@ -20,6 +20,8 @@
#include "legacy/ngraph_ops/fully_connected.hpp"
#include "ngraph/opsets/opset7.hpp"
#include "ngraph/opsets/opset9.hpp"
#include "ops/gna_convolution.hpp"
#include "ops/gna_max_pool.hpp"
namespace ov {
namespace intel_gna {
@@ -173,8 +175,6 @@ public:
*/
static inline std::shared_ptr<Limitations> get_instance();
static bool is_transpose_2d(const std::vector<size_t>& shape);
static bool is_transpose_supported(const std::vector<size_t>& shape);
static size_t get_min_batch_to_fit_in_buffer(InferenceEngine::DataPtr input);
/**
@@ -202,6 +202,13 @@ public:
* @return true if supported
*/
static bool is_split_supported(const std::shared_ptr<ov::Node>& node, bool is_exception_allowed = false);
/**
* @brief Validates if transpose is supported by GNA
* @param shape transpose
* @return true if supported
*/
static bool is_transpose_supported(const ov::Shape& shape);
/**
* @brief Validates if transpose is supported by GNA
* @param node transpose
@@ -209,13 +216,13 @@ public:
*/
static bool is_transpose_supported(const std::shared_ptr<const ov::Node>& node);
/**
* @brief Validates if legacy convolution is supported by GNA
* @param conv_ie convolution
* @brief Validates if convolution is supported by GNA
* @param conv_gna GNA convolution
* @param gna_precision GNA inference precision
* @param is_exception_allowed flag specifies whether exception is allowed
* @return true if supported
*/
bool is_conv_supported(const std::shared_ptr<ngraph::op::ConvolutionIE>& conv_ie,
bool is_conv_supported(const std::shared_ptr<ov::intel_gna::op::GNAConvolution>& conv_gna,
const InferenceEngine::Precision gna_precision,
bool is_exception_allowed = false);
/**
@@ -224,9 +231,19 @@ public:
* @param is_exception_allowed flag specifies whether exception is allowed
* @return true if precision is found in supported
*/
bool is_pooling_supported(const std::shared_ptr<ngraph::opset7::MaxPool> max_pool,
bool is_pooling_supported(const std::shared_ptr<ov::intel_gna::op::GNAMaxPool> max_pool,
bool is_exception_allowed = false);
static bool is_concat_supported(const std::shared_ptr<const ov::Node>& node);
static bool is_forward_transposed_concat_supported(const std::shared_ptr<const ov::Node>& node,
const AxisVector& order);
static bool is_backward_transposed_concat_supported(const std::shared_ptr<const ov::Node>& node,
const AxisVector& order);
static bool is_forward_transposed_split_supported(const std::shared_ptr<const ov::Node>& node,
const AxisVector& order);
static bool is_backward_transposed_split_supported(const std::shared_ptr<const ov::Node>& node,
const AxisVector& order);
/**
* @brief Validates if operation is supported by GNA
* @param node operation

View File

@@ -198,7 +198,8 @@ inline bool is_eltwise_add(const std::shared_ptr<ngraph::Node>& node) {
}
inline bool is_pooling(const std::shared_ptr<ngraph::Node>& node) {
return (std::dynamic_pointer_cast<ngraph::opset7::MaxPool>(node) != nullptr);
return ((std::dynamic_pointer_cast<ngraph::opset7::MaxPool>(node) != nullptr) ||
std::dynamic_pointer_cast<ov::intel_gna::op::GNAMaxPool>(node) != nullptr);
}
template <typename T>
@@ -268,7 +269,7 @@ inline bool has_32bit_output(const std::shared_ptr<ngraph::Node>& node) {
return ((std::dynamic_pointer_cast<ngraph::op::FullyConnected>(node) != nullptr) ||
(std::dynamic_pointer_cast<ngraph::opset9::MatMul>(node) != nullptr) ||
(std::dynamic_pointer_cast<ngraph::opset9::Convolution>(node) != nullptr) ||
(std::dynamic_pointer_cast<ngraph::op::ConvolutionIE>(node) != nullptr) ||
(std::dynamic_pointer_cast<ov::intel_gna::op::GNAConvolution>(node) != nullptr) ||
(std::dynamic_pointer_cast<ngraph::opset9::Add>(node) != nullptr) ||
(std::dynamic_pointer_cast<ngraph::opset9::Multiply>(node) != nullptr) ||
(std::dynamic_pointer_cast<ngraph::op::Eltwise>(node) != nullptr) ||
@@ -625,6 +626,20 @@ bool has_child_node(std::shared_ptr<ov::Node> node) {
return false;
}
/**
* @brief Checks if shape without dimensions == 1 is 2D
*/
inline bool is_shape_2d(const ov::Shape& shape) {
return graph_utils::squeeze_shape(shape).size() == 2;
}
/**
* @brief Checks if node has N consumers
*/
inline bool has_n_consumers(const std::shared_ptr<ov::Node>& node, size_t n_consumers) {
return node->output(0).get_target_inputs().size() == n_consumers;
}
} // namespace graph_utils
} // namespace intel_gna
} // namespace ov

View File

@@ -74,9 +74,11 @@ struct GnaDesc {
}
InferenceEngine::DataPtr to_ie_data() {
OPENVINO_SUPPRESS_DEPRECATED_START
return std::make_shared<InferenceEngine::Data>(
name,
InferenceEngine::TensorDesc(model_precision, dims, model_layout));
OPENVINO_SUPPRESS_DEPRECATED_END
}
};
@@ -98,9 +100,11 @@ struct InputDesc : GnaDesc {
}
InferenceEngine::InputInfo::Ptr ToIEInputInfo() {
OPENVINO_SUPPRESS_DEPRECATED_START
InferenceEngine::InputInfo::Ptr input_info = std::make_shared<InferenceEngine::InputInfo>();
input_info->setInputData(this->to_ie_data());
return input_info;
OPENVINO_SUPPRESS_DEPRECATED_END
}
};

View File

@@ -90,7 +90,7 @@ size_t LayerQuantizer::GetBiasSizeForLayer(InferenceEngine::WeightableLayer& wl)
return wl._biases->size();
} else if (LayerInfo(wl).isConvolution()) {
// Calculating biases len using outdata dims: biases number should be equal to output channels number
return InferenceEngine::GetDataDimByName(wl.outData.front(), InferenceEngine::DataDimName::C);
return InferenceEngine::GetDataDimSizeNHWC(wl.outData.front(), InferenceEngine::DataDimName::C);
} else {
// Calculating biases size using outData dimensions
return wl.outData.front()->getDims().back();

View File

@@ -1265,7 +1265,7 @@ bool ScaleFactorCalculator::ScaleFactorPerLayerWeightable(InferenceEngine::Weigh
double weights_reducer = 1.0;
auto conv = dynamic_cast<InferenceEngine::ConvolutionLayer*>(wl);
if (conv && !LayerInfo(conv).isConvolutionFilter()) {
const auto inDepth = GetDataDimByName(conv->insData.front().lock(), InferenceEngine::DataDimName::C);
const auto inDepth = GetDataDimSizeNHWC(conv->insData.front().lock(), InferenceEngine::DataDimName::C);
weights_reducer = gna_convolution_layer::getWeightsReducer(*conv);
weights_reducer *= MAX_VAL_2B_FEAT * scaleRange * inDepth / std::numeric_limits<int32_t>::max();
weights_reducer = std::max(1.0, weights_reducer);

View File

@@ -307,12 +307,26 @@ void GNAGraphCompiler::ConstPrimitive(InferenceEngine::CNNLayerPtr constLayer) {
void GNAGraphCompiler::assertConvolutionLayoutProper(const InferenceEngine::DataPtr& data) {
if (data->getLayout() != InferenceEngine::Layout::NHWC && data->getLayout() != InferenceEngine::Layout::NCHW &&
data->getLayout() != InferenceEngine::Layout::NC) {
data->getLayout() != InferenceEngine::Layout::NC && data->getLayout() != InferenceEngine::Layout::CHW) {
THROW_GNA_EXCEPTION << "layer: \"Convolution\" with layout " << data->getLayout()
<< " isn't currently supported on GNA";
}
}
namespace {
template <typename T>
PropertyVector<T> property_vector_append(PropertyVector<T> properties, T value) {
std::vector<T> new_values;
for (size_t i = 0; i < properties.size(); ++i)
new_values.push_back(properties[i]);
new_values.push_back(value);
return PropertyVector<T>(new_values);
}
} // namespace
/**
* Create AMIntelDNN Convolutional1DComponent from ConvolutionLayer
*
@@ -338,15 +352,24 @@ void GNAGraphCompiler::ConvolutionPrimitive(InferenceEngine::CNNLayerPtr layer)
const auto outputs = layer->outData.front();
assertConvolutionLayoutProper(inputs);
const auto in_batch = InferenceEngine::GetDataDimByName(inputs, InferenceEngine::DataDimName::N);
const auto in_channels = InferenceEngine::GetDataDimByName(inputs, InferenceEngine::DataDimName::C);
auto in_height = InferenceEngine::GetDataDimByName(inputs, InferenceEngine::DataDimName::H);
auto in_width = InferenceEngine::GetDataDimByName(inputs, InferenceEngine::DataDimName::W);
const auto in_batch = GetDataDimSizeNHWC(inputs, InferenceEngine::DataDimName::N);
const auto in_channels = GetDataDimSizeNHWC(inputs, InferenceEngine::DataDimName::C);
auto in_height = GetDataDimSizeNHWC(inputs, InferenceEngine::DataDimName::H);
auto in_width = GetDataDimSizeNHWC(inputs, InferenceEngine::DataDimName::W);
const auto out_batch = GetDataDimSizeNHWC(outputs, InferenceEngine::DataDimName::N);
const auto out_channels = GetDataDimSizeNHWC(outputs, InferenceEngine::DataDimName::C);
auto out_height = GetDataDimSizeNHWC(outputs, InferenceEngine::DataDimName::H);
auto out_width = GetDataDimSizeNHWC(outputs, InferenceEngine::DataDimName::W);
const auto out_batch = InferenceEngine::GetDataDimByName(outputs, InferenceEngine::DataDimName::N);
const auto out_channels = InferenceEngine::GetDataDimByName(outputs, InferenceEngine::DataDimName::C);
auto out_height = InferenceEngine::GetDataDimByName(outputs, InferenceEngine::DataDimName::H);
auto out_width = InferenceEngine::GetDataDimByName(outputs, InferenceEngine::DataDimName::W);
if (inputs->getLayout() == InferenceEngine::Layout::CHW) {
// convolution is ngraph-3D here. Make some fixes to work with it as it's ngraph-4D
convolution._kernel_y = 1;
convolution._dilation_y = 1;
convolution._stride_y = 1;
convolution._padding = property_vector_append<unsigned int>(convolution._padding, 0);
convolution._pads_end = property_vector_append<unsigned int>(convolution._pads_end, 0);
}
if (in_height > 1 && in_width == 1 && !ShouldUseOnlyConv2DGnaIface()) {
std::swap(in_height, in_width);
@@ -589,42 +612,12 @@ void GNAGraphCompiler::finalizeConvolution1DPrimitive(InferenceEngine::CNNLayerP
});
}
// TODO: convolution might be not the first layer in sorted order but connected via split for example - dont know
// how kaldi will handle that
if (!dnn->do_rotate_input) {
if ((inputs->getLayout() != InferenceEngine::Layout::NHWC || transpose_h_w) &&
LayerInfo(connectedInputLayer).isInput()) {
// Kaldi features are opposite orientation
dnn->do_rotate_input = true;
dnn->num_rotate_rows = effectiveStride;
dnn->num_rotate_columns = num_inputs / effectiveStride;
} else {
dnn->do_rotate_input = false;
}
}
connectOutput(layer, ptr_outputs, num_data_bytes_out);
// Transpose H with W or C with HW
auto A = transpose_h_w ? in_kernel_h : in_channels;
auto B = transpose_h_w ? in_kernel_w : convolution._kernel[X_AXIS];
std::vector<uint8_t> transposedWeights;
for (uint32_t k = 0; k < num_filters; k++) {
uint8_t* ptr_filt_current =
convolution._weights->cbuffer().as<uint8_t*>() + k * A * B * convolution.precision.size();
auto transposedPart = transposeMatrix(ptr_filt_current, convolution.precision.size(), A, B);
transposedWeights.insert(transposedWeights.end(), transposedPart.begin(), transposedPart.end());
}
if (transposedWeights.size() != convolution._weights->byteSize()) {
THROW_GNA_LAYER_EXCEPTION(&convolution) << "weights was transposed incorrectly. " << transposedWeights.size()
<< ' ' << convolution._weights->byteSize();
}
if (num_conv_kernel_padding == 0) {
gnamem->getQueue(REGION_RO)->push_local_ptr(layer,
ptr_weights,
transposedWeights.data(),
convolution._weights->cbuffer(),
convolution._weights->byteSize());
} else {
auto paddedWeights = num_filter_coefficients * num_filters;
@@ -636,7 +629,7 @@ void GNAGraphCompiler::finalizeConvolution1DPrimitive(InferenceEngine::CNNLayerP
layerName,
num_conv_kernel_padding,
cpSize,
transposedWeights,
convolution,
num_filters,
single_conv_kernel_size](void* data, std::size_t size) {
if (paddedWeightsSize > size) {
@@ -648,7 +641,7 @@ void GNAGraphCompiler::finalizeConvolution1DPrimitive(InferenceEngine::CNNLayerP
for (uint32_t i = 0; i < num_filters; i++) {
ie_memcpy(dstPtr + offset,
size - offset,
transposedWeights.data() + single_conv_kernel_size * i * cpSize,
convolution._weights->cbuffer().as<uint8_t*>() + single_conv_kernel_size * i * cpSize,
single_conv_kernel_size * cpSize);
offset += single_conv_kernel_size * cpSize;
ie_memcpy(dstPtr + offset, size - offset, &padding_zeros[0], padding_zeros.size());
@@ -783,22 +776,6 @@ void GNAGraphCompiler::finalizeConvolution2DPrimitive(InferenceEngine::CNNLayerP
auto connectedInputLayer = connectInput(layer, ptr_inputs, num_data_bytes_in).input;
// TODO: convolution might be not the first layer in sorted order but connected via split for example - dont know
// how kaldi will handle that
if (!dnn->do_rotate_input && inputs->getLayout() != InferenceEngine::Layout::NHWC &&
LayerInfo(connectedInputLayer).isInput()) {
// Kaldi features are opposite orientation
dnn->do_rotate_input = true;
dnn->num_rotate_rows = in_channels;
if (in_height != 1) {
dnn->num_rotate_rows *= convolution._stride_y;
}
if (in_width != 1) {
dnn->num_rotate_rows *= convolution._stride_x;
}
dnn->num_rotate_columns = num_inputs / dnn->num_rotate_rows;
}
connectOutput(layer, ptr_outputs, num_data_bytes_out);
const auto convolution_precision = convolution.precision.size();
@@ -815,7 +792,7 @@ void GNAGraphCompiler::finalizeConvolution2DPrimitive(InferenceEngine::CNNLayerP
ALIGN(effective_single_kernel_size, Limitations::kConvEachKernelByteAlignment) - effective_single_kernel_size;
for (uint32_t k = 0; k < convolution._out_depth; k++) {
uint8_t* ptr_filt_current = convolution._weights->cbuffer().as<uint8_t*>() + k * single_kernel_size;
auto transposed_part = transposeMatrix(ptr_filt_current, convolution_precision, in_channels, kernelHW);
auto transposed_part = copy_matrix(ptr_filt_current, convolution.precision.size(), in_channels, kernelHW);
transposed_weights.insert(transposed_weights.end(), transposed_part.begin(), transposed_part.end());
transposed_weights.resize(transposed_weights.size() + effective_single_kernel_size - single_kernel_size +
kernel_pad);
@@ -997,13 +974,19 @@ void GNAGraphCompiler::PoolingPrimitive(InferenceEngine::CNNLayerPtr layer) {
auto inputs = layer->insData.begin()->lock();
auto outputs = *layer->outData.begin();
uint32_t w_dim_in = InferenceEngine::GetDataDimByName(inputs, InferenceEngine::DataDimName::W);
uint32_t h_dim_in = InferenceEngine::GetDataDimByName(inputs, InferenceEngine::DataDimName::H);
const uint32_t c_dim_in = InferenceEngine::GetDataDimByName(inputs, InferenceEngine::DataDimName::C);
uint32_t w_dim_in = GetDataDimSizeNHWC(inputs, InferenceEngine::DataDimName::W);
uint32_t h_dim_in = GetDataDimSizeNHWC(inputs, InferenceEngine::DataDimName::H);
const uint32_t c_dim_in = GetDataDimSizeNHWC(inputs, InferenceEngine::DataDimName::C);
uint32_t w_dim_out = InferenceEngine::GetDataDimByName(outputs, InferenceEngine::DataDimName::W);
uint32_t h_dim_out = InferenceEngine::GetDataDimByName(outputs, InferenceEngine::DataDimName::H);
const uint32_t c_dim_out = InferenceEngine::GetDataDimByName(outputs, InferenceEngine::DataDimName::C);
uint32_t w_dim_out = GetDataDimSizeNHWC(outputs, InferenceEngine::DataDimName::W);
uint32_t h_dim_out = GetDataDimSizeNHWC(outputs, InferenceEngine::DataDimName::H);
const uint32_t c_dim_out = GetDataDimSizeNHWC(outputs, InferenceEngine::DataDimName::C);
if (inputs->getLayout() == InferenceEngine::Layout::CHW) {
// Pooling is ngraph-3D here. Make some fixes to work with it as it's ngraph-4D
pooling._kernel = property_vector_append<unsigned int>(pooling._kernel, 1);
pooling._stride = property_vector_append<unsigned int>(pooling._stride, 1);
}
void* ptr_inputs = nullptr;
void* ptr_outputs = nullptr;
@@ -2590,7 +2573,8 @@ void GNAGraphCompiler::connectOutput(InferenceEngine::CNNLayerPtr layer, void* p
if (layer->params.find("output_offset") != layer->params.end()) {
output_offset = layer->GetParamAsInt("output_offset");
}
gnamem->getQueue(REGION_AUTO)->bind_ptr(layer, ptr, &concatLayerInfoItem.gna_ptr, output_offset);
gnamem->getQueue(REGION_AUTO)
->bind_ptr(layer, ptr, &concatLayerInfoItem.gna_ptr, output_offset, num_data_bytes_out);
}
return;
}
@@ -2859,5 +2843,15 @@ std::vector<uint8_t> GNAGraphCompiler::transposeMatrix(uint8_t* ptr_matrix,
return temp_buffer;
}
std::vector<uint8_t> GNAGraphCompiler::copy_matrix(uint8_t* ptr_matrix,
size_t element_size,
uint32_t num_rows,
uint32_t num_cols) {
const size_t dest_size = num_rows * num_cols * element_size;
std::vector<uint8_t> temp_buffer(dest_size);
::memcpy(temp_buffer.data(), ptr_matrix, dest_size);
return temp_buffer;
}
} // namespace intel_gna
} // namespace ov

View File

@@ -53,6 +53,10 @@ private:
size_t element_size,
uint32_t num_rows,
uint32_t num_cols);
std::vector<uint8_t> static copy_matrix(uint8_t* ptr_matrix,
size_t element_size,
uint32_t num_rows,
uint32_t num_cols);
bool ShouldUseOnlyConv2DGnaIface() const;

View File

@@ -237,32 +237,6 @@ inline InferenceEngine::CNNLayerPtr FindPermutationAfterConvolutionInKaldiModel(
return next;
}
/**
* @brief identifies if a model must be converted to NHWC, it must not be neither NHWC, nor Kaldi
* @param layers model sorted layers
*/
inline bool MustBeConvertedFromNCHWToNHWC(const std::vector<InferenceEngine::CNNLayerPtr>& layers) {
for (auto& l : layers) {
if (!LayerInfo(l).isConvolution())
continue;
InferenceEngine::CNNLayerPtr next;
std::tie(std::ignore, next) = FindPermutationsAroundConvolutionInNHWCModel(l);
if (next != nullptr)
return false;
// If a convolution has only 1-dimension input and output we should skip it
auto in_dims = l->insData.begin()->lock()->getDims();
auto out_dims = l->outData.front()->getDims();
if (ov::intel_gna::graph_utils::is_one_dim_shapes(in_dims, out_dims)) {
continue;
}
return FindPermutationAfterConvolutionInKaldiModel(l) == nullptr;
}
return false;
}
/**
* @brief returns transposition information for a layer based on the previous convolution or pooling dimensions order
* @param layer layer from which transposition info search must be started

View File

@@ -924,4 +924,38 @@ inline uint32_t GetDataDimByName(InferenceEngine::DataPtr data, DataDimName dimN
return GetDimFromBack(dims, backOffsets[dimIxInNCHW]);
}
/**
* @brief returns a size of a specified data dimension depending on the layout
* NHWC specialization
* @param data a pointer to the data
* @param dimName dimension name
*/
inline uint32_t GetDataDimSizeNHWC(InferenceEngine::DataPtr data, DataDimName dimName) {
uint32_t dimIxInNCHW = static_cast<uint32_t>(dimName);
IE_ASSERT(dimIxInNCHW <= 3);
std::vector<uint32_t> backOffsets;
switch (data->getLayout()) {
case Layout::C:
case Layout::NC:
// 1 will be returned for offsets > 2
backOffsets = std::vector<uint32_t>{2, 1, 3, 4};
break;
case Layout::HWC:
// 1 will be returned for offset 4
case Layout::NHWC:
backOffsets = std::vector<uint32_t>{4, 3, 2, 1};
break;
case Layout::CHW:
// 1 will be returned for offset 4
case Layout::NCHW:
backOffsets = std::vector<uint32_t>{4, 1, 3, 2};
break;
default:
THROW_GNA_EXCEPTION << data->getName() << " Unexpected layout " << data->getLayout();
}
auto dims = data->getDims();
return GetDimFromBack(dims, backOffsets[dimIxInNCHW]);
}
} // namespace InferenceEngine

View File

@@ -344,13 +344,13 @@ void GNAPlugin::PrePostProcess(InferenceEngine::Blob::Ptr input_blob,
std::shared_ptr<ov::Model> model) {
const ov::element::Type input_type = details::convertPrecision(input_blob->getTensorDesc().getPrecision());
const ov::element::Type output_type = details::convertPrecision(output_blob->getTensorDesc().getPrecision());
const ov::Shape& input_shape = model->get_parameters().front()->get_shape();
const ov::Shape& output_shape = model->get_results().front()->get_shape();
const ov::Shape& output_shape = output_blob->getTensorDesc().getDims();
for (const auto& param : model->get_parameters()) {
param->set_element_type(input_type);
}
model->validate_nodes_and_infer_types();
const ov::Shape& input_shape = model->get_parameters()[0]->get_output_shape(0);
ov::TensorVector inputs = {ov::Tensor(input_type, input_shape, input_blob->cbuffer().as<void*>())};
ov::TensorVector results = {ov::Tensor(output_type, output_shape, output_blob->buffer().as<void*>())};
@@ -611,52 +611,6 @@ bool GNAPlugin::TryToInitOutput(const std::string& portName, InferenceEngine::CN
return false;
}
void GNAPlugin::FillInputsAndOutputsTranspositionInfo(const InferenceEngine::CNNNetwork& net) {
OV_ITT_SCOPED_TASK(itt::domains::GNA_LT, "FillInputsAndOutputsTranspositionInfo");
auto printTranspositionInfo = [](const std::vector<TranspositionInfo>& transpositionInfo) {
for (const auto& transpositionInfoPart : transpositionInfo) {
log::debug() << "transpose=" << transpositionInfoPart.transpose
<< " rows_num=" << transpositionInfoPart.num_transpose_rows
<< " columns_num=" << transpositionInfoPart.num_transpose_columns << "\n";
}
};
auto inputLayers = CNNNetGetAllInputLayers(net);
for (const auto& inputLayer : inputLayers) {
// Collect information for inputs transposition
if (!LayerInfo(inputLayer).isInput())
continue;
auto transpositionInfo = FindTranspositionInfoFromNextLayers(inputLayer);
if (transpositionInfo.empty())
continue;
transpose_inputs_info.insert({inputLayer->name, transpositionInfo});
log::debug() << "Input " << inputLayer->name << " transposition info: \n";
printTranspositionInfo(transpositionInfo);
}
auto outputsMap = net.getOutputsInfo();
for (const auto& outPort : outputsMap) {
auto outLayer = getCreatorLayer(outPort.second).lock();
// Collect information for outputs transposition
if (!LayerInfo(outLayer).isOutput())
continue;
auto transpositionInfo = FindTranspositionInfoFromPrevLayers(outLayer);
if (transpositionInfo.empty())
continue;
// Swap transposition info rows and columns since we need to transpose output back from NHWC to NCHW
for (auto&& transpositionInfoPart : transpositionInfo) {
if (transpositionInfoPart.transpose) {
std::swap(transpositionInfoPart.num_transpose_rows, transpositionInfoPart.num_transpose_columns);
}
}
transpose_outputs_info.insert({outLayer->name, transpositionInfo});
log::debug() << "Output " << outLayer->name << " transposition info: \n";
printTranspositionInfo(transpositionInfo);
}
}
#ifdef PLOT
void GNAPlugin::AddDebugProperties(const InferenceEngine::CNNLayerPtr layer,
InferenceEngine::ordered_properties& printed_properties,
@@ -751,10 +705,6 @@ void GNAPlugin::LoadNetwork(const CNNNetwork& _network) {
UpdateInputScaleFromNetwork(network);
}
if (MustBeConvertedFromNCHWToNHWC(CNNNetSortTopologically(network))) {
FillInputsAndOutputsTranspositionInfo(network);
}
InferenceEngine::CNNNetwork newNet;
if (gnaFlags->sw_fp32) {
@@ -995,22 +945,6 @@ void GNAPlugin::LoadNetwork(const CNNNetwork& _network) {
}
}
if (dnn->do_rotate_input && transpose_inputs_info.empty()) {
for (auto& inputLayer : inputLayers) {
transpose_inputs_info.insert(
{inputLayer->name,
{TranspositionInfo{dnn->do_rotate_input, dnn->num_rotate_rows, dnn->num_rotate_columns}}});
}
}
// TODO: Need to remove this conversion when ngraph NCHW->NHWC transformation is enabled
if (!transpose_inputs_info.empty()) {
ConvertTransposeMapToModel(transpose_inputs_info, inputs_ptr_->Get());
}
if (!transpose_outputs_info.empty()) {
ConvertTransposeMapToModel(transpose_outputs_info, outputs_.Get());
}
DumpXNNToFile();
#ifdef PLOT

View File

@@ -37,6 +37,8 @@ class WorkerPool;
class Worker;
} // namespace request
using namespace ov::intel_gna::pre_post_processing;
class GNAPlugin : public InferenceEngine::IInferencePlugin {
protected:
std::string _pluginName = "GNA";
@@ -204,6 +206,13 @@ protected:
InferenceEngine::Blob::Ptr output_blob,
std::shared_ptr<ov::Model> model);
/**
* Run ngraph model on CPU to modify inputs/outputs
*/
void pre_post_process(InferenceEngine::Blob::Ptr input_blob,
InferenceEngine::Blob::Ptr output_blob,
std::shared_ptr<ov::Model> model);
void ImportFrames(void* ptr_dst,
const void* ptr_src,
InferenceEngine::Precision input_precision,
@@ -246,14 +255,6 @@ protected:
* @return true if the output is initiated, false otherwise
*/
bool TryToInitOutput(const std::string& portName, InferenceEngine::CNNLayerPtr layer);
/**
* @brief Fills inputs and outputs transposition info for model convertion from NCHW to NHWC.
* Information for transposition is found from convolution/pooling input or output dimensions.
* @param layers model sorted layers
*/
void FillInputsAndOutputsTranspositionInfo(const InferenceEngine::CNNNetwork& net);
bool isFP32ModeActive() const;
std::shared_ptr<request::ModelWrapper> createModelWrapperForLoadNetwork(bool trivial);
std::shared_ptr<request::ModelWrapper> createModelWrapperForImportNetwork(uint32_t numberOfOperations);

View File

@@ -7,6 +7,8 @@
#include "gna_itt.hpp"
#include "legacy/net_pass.h"
#include "legacy/transformations/convert_opset1_to_legacy/convert_opset1_to_legacy.hpp"
#include "legacy/transformations/convert_opset1_to_legacy/convert_strided_slice_to_crop.hpp"
#include "ngraph/opsets/opset2.hpp"
#include "ngraph/opsets/opset7.hpp"
#include "openvino/pass/manager.hpp"
#include "optimizer/gna_pass_manager.hpp"
@@ -18,7 +20,9 @@
#include "transformations/common_optimizations/fq_reshape_fusion.hpp"
#include "transformations/common_optimizations/pull_transpose_through_fq.hpp"
#include "transformations/common_optimizations/relu_fake_quantize_fusion.hpp"
#include "transformations/common_optimizations/reshape_sequence_fusion.hpp"
#include "transformations/common_optimizations/transpose_sinking.hpp"
#include "transformations/common_optimizations/transpose_to_reshape.hpp"
#include "transformations/control_flow/unroll_tensor_iterator.hpp"
#include "transformations/convert_dwsc_to_scaleshifts.hpp"
#include "transformations/convert_matmul_to_pointwise_convolution.hpp"
@@ -28,6 +32,8 @@
#include "transformations/decompose_mvn.hpp"
#include "transformations/fp16_compression/convert_compression_only_to_legacy.hpp"
#include "transformations/fp16_compression/mark_decompression_convert_constant_folding.hpp"
#include "transformations/fuse_conv_bias_activation.hpp"
#include "transformations/gather_sinking.hpp"
#include "transformations/handle_transposes_around_matmul.hpp"
#include "transformations/init_node_info.hpp"
#include "transformations/insert_copy_layer.hpp"
@@ -37,6 +43,7 @@
#include "transformations/markup_fusable_transpose.hpp"
#include "transformations/op_conversions/convert_mvn1_to_mvn6.hpp"
#include "transformations/op_conversions/convert_sequences_to_tensor_iterator.hpp"
#include "transformations/op_conversions/convert_slice_to_strided_slice.hpp"
#include "transformations/op_conversions/gru_cell_decomposition.hpp"
#include "transformations/op_conversions/lstm_cell_decomposition.hpp"
#include "transformations/op_conversions/softsign_decomposition.hpp"
@@ -48,13 +55,28 @@
#include "transformations/remove_in_out_processing.hpp"
#include "transformations/remove_single_input_concat.hpp"
#include "transformations/reorder_activation_and_pooling.hpp"
#include "transformations/replace_gna_nhwc_layers.hpp"
#include "transformations/reshape_transpose_substitute.hpp"
#include "transformations/rotate_inputs.hpp"
#include "transformations/split_convolution_with_large_buffer_size.hpp"
#include "transformations/split_eltwise.hpp"
#include "transformations/substitute_softsign.hpp"
#include "transformations/swap_input_matmul_gna.hpp"
#include "transformations/transpose_sinking/ts_concat.hpp"
#include "transformations/transpose_sinking/ts_fuse.hpp"
#include "transformations/transpose_sinking/ts_general.hpp"
#include "transformations/transpose_sinking/ts_split.hpp"
#include "transformations/ts_concat_forward.hpp"
#include "transformations/ts_split_backward.hpp"
#include "transformations/unfuse_reshape_and_transpose.hpp"
#include "transformations/utils/transformation_helper.hpp"
#include "transformations/utils/utils.hpp"
using namespace ov;
using namespace ov::opset8;
using namespace ov::intel_gna::limitations;
using namespace ov::intel_gna::pass::helper;
namespace ov {
namespace intel_gna {
@@ -64,12 +86,13 @@ void TransformationsPipeline::apply(const std::shared_ptr<ov::Model>& model,
fake_quantized = ov::op::util::has_op_with_type<ngraph::op::FakeQuantize>(model);
const bool has_convolution = ov::op::util::has_op_with_type<ngraph::opset7::Convolution>(model);
const bool has_maxpool = ov::op::util::has_op_with_type<ov::opset8::MaxPool>(model);
const bool has_slice = ov::op::util::has_op_with_type<ov::opset8::Slice>(model);
const bool has_matmul = ov::op::util::has_op_with_type<ngraph::opset7::MatMul>(model);
const bool has_mvn = ov::op::util::has_op_with_type<ngraph::opset7::MVN>(model) ||
const bool has_mvn = ov::op::util::has_op_with_type<ov::opset8::MVN>(model) ||
ov::op::util::has_op_with_type<ov::op::v0::MVN>(model);
ov::pass::Manager manager;
manager.register_pass<ov::pass::InitNodeInfo>();
// In OV API 2.0(IRv10) default convertion to fp32 (inputs, outputs and weights) is disabled
// and we need to run the ConvertPrecision transformation to support old networks.
manager.register_pass<ov::pass::ConvertPrecision>(precisions_map{{ngraph::element::f16, ngraph::element::f32}});
@@ -104,7 +127,6 @@ void TransformationsPipeline::apply(const std::shared_ptr<ov::Model>& model,
manager.register_pass<ov::intel_gna::pass::SwapInputMatMulWithBias>();
manager.register_pass<ov::intel_gna::pass::SwapInputMatMul>();
manager.register_pass<ov::intel_gna::pass::HandleTransposesAroundMatMul>();
manager.register_pass<ov::intel_gna::pass::InsertTransposeAfterConvOrPool>();
manager.register_pass<ov::intel_gna::pass::Unfuse2dto4dReshapeAndTranspose>();
manager.register_pass<ov::intel_gna::pass::Unfuse4dto2dReshapeAndTranspose>();
manager.register_pass<ov::intel_gna::pass::RemoveExtraReshapes>();
@@ -112,11 +134,21 @@ void TransformationsPipeline::apply(const std::shared_ptr<ov::Model>& model,
manager.register_pass<ov::intel_gna::pass::RemoveSingleInputConcat>();
manager.register_pass<ov::intel_gna::pass::SubstituteSoftsign>();
manager.register_pass<ov::intel_gna::pass::InsertCopyBeforeLayerToBeEliminated>();
if (!has_convolution && !has_matmul && !has_mvn) {
// TODO: Remove this condition when the legacy layout transformation (NCHW->NHWC) is disabled
manager.register_pass<ov::intel_gna::pass::RemoveInputsProcessing>(input_output_subgraphs);
manager.register_pass<ov::intel_gna::pass::RemoveOutputsProcessing>(input_output_subgraphs);
// TODO enable this transformation for networks without convolutions
if (has_convolution || has_maxpool || has_mvn || has_matmul) {
manager.register_pass<ov::intel_gna::pass::ReplaceGnaNHWCLayers>();
manager.register_pass<ov::intel_gna::pass::InsertConvolutionTransposeHW>();
manager.register_pass<ov::pass::TransposeSinkingGeneral>();
manager.register_pass<ov::intel_gna::pass::GatherSinkingGeneral>();
manager.register_pass<ov::pass::ReshapeSequenceFusion>();
manager.register_pass<ov::pass::TransposeToReshape>();
manager.register_pass<ov::intel_gna::pass::GnaConvolutionFusion>();
manager.register_pass<ov::intel_gna::pass::TSConcatForward>();
manager.register_pass<ov::intel_gna::pass::TSSplitBackward>();
manager.register_pass<ov::pass::transpose_sinking::TSFuse>();
}
manager.register_pass<ov::intel_gna::pass::RemoveInputsProcessing>(input_output_subgraphs);
manager.register_pass<ov::intel_gna::pass::RemoveOutputsProcessing>(input_output_subgraphs);
manager.register_pass<ov::pass::ConvertOpSet3ToOpSet2>();
manager.register_pass<ov::pass::ConvertOpSet2ToOpSet1>();
manager.register_pass<ngraph::pass::ConvertOpSet1ToLegacy>();
@@ -160,6 +192,62 @@ void TransformationsPipeline::apply(const std::shared_ptr<ov::Model>& model,
{ov::element::u32, ov::element::i32}});
const auto& pass_config = manager.get_pass_config();
pass_config->set_callback<ov::pass::transpose_sinking::TSConcatForward>(
[](const std::shared_ptr<const ov::Node>& node) -> bool {
const TransposeInfo transpose_info = get_first_input_transpose(node);
if (transpose_info.isEmpty())
return false;
const bool is_supported = Limitations::is_forward_transposed_concat_supported(
node,
transpose_info.transpose_const->get_axis_vector_val());
if (!is_supported)
mark_input_transposes_as_nosinking(node);
return !is_supported;
});
pass_config->set_callback<ov::pass::transpose_sinking::TSConcatBackward>(
[](const std::shared_ptr<const ov::Node>& node) -> bool {
const TransposeInfo transpose_info = get_first_output_transpose(node);
if (transpose_info.isEmpty())
return false;
return !Limitations::is_backward_transposed_concat_supported(
node,
transpose_info.transpose_const->get_axis_vector_val());
});
pass_config->set_callback<ov::pass::transpose_sinking::TSSplitForward>(
[](const std::shared_ptr<const ov::Node>& node) -> bool {
const TransposeInfo transpose_info = get_first_input_transpose(node);
if (transpose_info.isEmpty())
return false;
const bool is_supported = Limitations::is_forward_transposed_split_supported(
node,
transpose_info.transpose_const->get_axis_vector_val());
if (!is_supported)
mark_input_transposes_as_nosinking(node);
return !is_supported;
});
pass_config->set_callback<ov::pass::transpose_sinking::TSSplitBackward>(
[](const std::shared_ptr<const ov::Node>& node) -> bool {
const TransposeInfo transpose_info = get_first_output_transpose(node);
if (transpose_info.isEmpty())
return false;
return !Limitations::is_backward_transposed_split_supported(
node,
transpose_info.transpose_const->get_axis_vector_val());
});
/**
* TransposeSinking doesn't currently support StridedSlice. We disable SliceToStridedSlice
* transformation to prevent convert Slice to StridedSlice. This allows us to work with
* networks, that initialy have Slice.
* That could be removed after StridedSlice implementation in TransposeSinking
*/
if (has_slice && (has_convolution || has_maxpool || has_mvn)) {
pass_config->disable<ov::pass::SliceToStridedSlice>();
}
// Allowing FP16 Converts to be folded and FP16 constants to upgrade to FP32 data type
pass_config->disable<ov::pass::ConvertCompressedOnlyToLegacy>();
pass_config->disable<ov::pass::DisableDecompressionConvertConstantFolding>();
@@ -177,8 +265,23 @@ void TransformationsPipeline::apply(const std::shared_ptr<ov::Model>& model,
// Operations Max and Min aren't supported
pass_config->disable<ov::pass::ConcatReduceFusion>();
pass_config->disable<ov::pass::ConcatReduceFusion>();
manager.run_passes(model);
/**
* As we disabled SliceToStridedSlice, we have after all transformations
* Slice, that is not supported natively in our plugin. Here we convert
* Slice -> StridedSlice -> CropIE
* That could be removed after StridedSlice implementation in TransposeSinking
*/
if (has_slice && (has_convolution || has_maxpool || has_mvn)) {
ov::pass::Manager manager;
manager.register_pass<ov::pass::InitNodeInfo>();
manager.register_pass<ov::pass::SliceToStridedSlice>(true);
manager.register_pass<ngraph::pass::ConvertStridedSliceToCropMatcher>();
manager.run_passes(model);
}
is_ngraph_passes_used = true;
}
@@ -204,8 +307,6 @@ void TransformationsPipeline::apply_legacy(const InferenceEngine::CNNNetwork& ne
passes->registerPass<FuseFQIntoWeightsPass>();
passes->registerPass<MoveFakeQuantizeLayerIntoQuantParamsPass>();
passes->registerPass<TransposeWeightsFromNCHWToNHWCPass>();
passes->registerPass<SubstitutePReluPass>();
if (!is_ngraph_passes_used) {
@@ -221,7 +322,7 @@ void TransformationsPipeline::apply_legacy(const InferenceEngine::CNNNetwork& ne
passes->registerPass<FlattenTrivialConcatPass>();
passes->registerPass<InsertConcatAligningFilterPass>();
passes->registerPass<ReorderConcatInputsPass>();
passes->registerPass<RemovePermutationsNHWCToNCHWPass>();
// Keep legacy inserting of Identity layer here
// because concat and split aliging passes are not moved to ngraph yet
passes->registerPass<InsertIdentityLayerPass>();

View File

@@ -59,11 +59,11 @@ double getWeightsReducer(InferenceEngine::ConvolutionLayer& conv) {
const std::vector<KRT> reducers{{49, 3.0}, {36, 2.6}, {21, 2.3}, {14, 1.7}, {9, 1.3}, {7, 1.2}};
auto reducer = 1.0;
const auto inDepth =
InferenceEngine::GetDataDimByName(conv.insData.front().lock(), InferenceEngine::DataDimName::C);
InferenceEngine::GetDataDimSizeNHWC(conv.insData.front().lock(), InferenceEngine::DataDimName::C);
const auto inHeight =
InferenceEngine::GetDataDimByName(conv.insData.front().lock(), InferenceEngine::DataDimName::H);
InferenceEngine::GetDataDimSizeNHWC(conv.insData.front().lock(), InferenceEngine::DataDimName::H);
const auto inWidth =
InferenceEngine::GetDataDimByName(conv.insData.front().lock(), InferenceEngine::DataDimName::W);
InferenceEngine::GetDataDimSizeNHWC(conv.insData.front().lock(), InferenceEngine::DataDimName::W);
if (is3DInputOr2DKernel(inHeight, inWidth, inDepth, conv._kernel_y, conv._kernel_x) &&
!isMappableFrom2DTo1D(inHeight,
inWidth,

View File

@@ -297,7 +297,7 @@ public:
return isOfType("FakeQuantize");
}
bool isNonFunctional() const {
return isOfType("reshape") || isOfType("squeeze") || isOfType("unsqueeze") || isTrivialPermute();
return isOfType("reshape") || isOfType("squeeze") || isOfType("unsqueeze") || isTrivialPermute() || is_gather();
}
bool isReshape() const noexcept {
return isOfType("reshape");
@@ -305,6 +305,9 @@ public:
bool isPermute() const noexcept {
return isOfType("permute");
}
bool is_gather() const noexcept {
return isOfType("gather");
}
bool isPermuteFusable() const noexcept {
return isPermute() &&
(layer->params.count(ov::intel_gna::rt_info::GNATransposeFusable::get_type_info_static()) > 0);
@@ -349,11 +352,8 @@ public:
bool isNonValuesChangable() const {
return isNonFunctional() || isSplit() || isSlice() || isConcat();
}
bool is_gather() const noexcept {
return isOfType("gather");
}
bool is_fq_non_sensitive() const {
return isPermute() || is_gather() || isNonFunctional();
return isPermute() || isNonFunctional();
}
bool isPooling() const noexcept {
return isOfType("pooling");

View File

@@ -133,6 +133,9 @@ public:
void set_auto_pad(const ov::op::PadType& auto_pad) {
m_auto_pad = auto_pad;
}
bool has_add_node() const {
return m_has_add_node;
}
bool has_bias() const {
return m_has_add_node;
}

View File

@@ -110,17 +110,6 @@ static void SumBlobs(Blob::Ptr& src_blob, Blob::Ptr& dst_blob) {
}
}
static Blob::Ptr convertToRWBlob(const Blob::Ptr& readOnlyBlob, const std::string& name = {}) {
auto blob = Blob::CreateFromData(std::make_shared<Data>(name, readOnlyBlob->getTensorDesc()));
blob->allocate();
const auto ret = ie_memcpy(blob->buffer().as<uint8_t*>(),
blob->size() * blob->getTensorDesc().getPrecision().size(),
readOnlyBlob->buffer().as<uint8_t*>(),
readOnlyBlob->size() * readOnlyBlob->getTensorDesc().getPrecision().size());
IE_ASSERT(ret == 0);
return blob;
}
// indexes stored in pass manager
static const char identityLayersCounterName[] = "identityLayerCounter";
static const char diagonalLayersCounterName[] = "diagonalLayerCounter";
@@ -2419,225 +2408,6 @@ void MoveFakeQuantizeLayerIntoQuantParamsPass ::run() {
}
}
void TransposeWeightsFromNCHWToNHWCPass::run() {
OV_ITT_SCOPED_TASK(itt::domains::GNA_LT, "TransposeWeightsFromNCHWToNHWCPass");
if (!MustBeConvertedFromNCHWToNHWC(*pLayers))
return;
auto printTranspositionInfo = [](const std::vector<TranspositionInfo>& transpositionInfo) {
for (const auto& transpositionInfoPart : transpositionInfo) {
log::debug() << "transpose=" << transpositionInfoPart.transpose
<< " rows_num=" << transpositionInfoPart.num_transpose_rows
<< " columns_num=" << transpositionInfoPart.num_transpose_columns << "\n";
}
};
auto transpInfoMatchWeightsSize =
[](const std::vector<TranspositionInfo>& transpositionInfo, size_t weightsSize, const std::string& layerName) {
size_t totalElements = 0;
for (auto&& transpositionInfoPart : transpositionInfo) {
totalElements += transpositionInfoPart.num_transpose_rows * transpositionInfoPart.num_transpose_columns;
}
if (totalElements != weightsSize) {
THROW_GNA_EXCEPTION << layerName << " weights elements from transposition info (" << totalElements
<< ") don't match input dimensions (" << weightsSize << ")";
}
};
for (auto&& l : *pLayers) {
if (LayerInfo(l).isScaleShift()) {
std::vector<TranspositionInfo> transpositionInfo;
// Try to find a convolution in previous layers
if (InferenceEngine::CNNNetHasPrevLayer(l.get())) {
transpositionInfo = FindTranspositionInfoFromPrevLayers(InferenceEngine::CNNNetPrevLayer(l));
// If no convolutions are found try to find them in next layers
if (!FoundPartToTranspose(transpositionInfo) && !l->outData.empty() &&
!getInputTo(l->outData[0]).empty()) {
transpositionInfo = FindTranspositionInfoFromNextLayers(getInputTo(l->outData[0]).begin()->second);
}
}
if (FoundPartToTranspose(transpositionInfo)) {
if (l->input()->getDims().front() > 1) {
THROW_GNA_EXCEPTION << l->name
<< " Weights transposition is not supported for a layer with batch size > 1";
}
auto weightable = dynamic_cast<WeightableLayer*>(l.get());
IE_ASSERT(weightable != nullptr);
size_t totalWeights = weightable->_weights->size();
transpInfoMatchWeightsSize(transpositionInfo, totalWeights, l->name);
ConvertTensorFromNCHWToNHWC(weightable->precision.size(),
1,
weightable->_weights->size(),
weightable->_weights->cbuffer().as<uint8_t*>(),
true,
transpositionInfo);
if (weightable->_biases) {
ConvertTensorFromNCHWToNHWC(weightable->precision.size(),
1,
weightable->_biases->size(),
weightable->_biases->cbuffer().as<uint8_t*>(),
true,
transpositionInfo);
}
log::debug() << l->name << " weights and biases rows transposition info:\n";
printTranspositionInfo(transpositionInfo);
}
}
if (LayerInfo(l).isFullyConnected()) {
auto weightable = dynamic_cast<WeightableLayer*>(l.get());
IE_ASSERT(weightable != nullptr);
IE_ASSERT(weightable->_weights != nullptr);
auto precision = weightable->precision.size();
auto out_dims = l->outData[0]->getDims();
auto in_dims = l->input()->getDims();
auto weightsRows = InferenceEngine::details::product(std::begin(out_dims) + 1, std::end(out_dims));
auto weightsColumns = InferenceEngine::details::product(std::begin(in_dims) + 1, std::end(in_dims));
// Find a convolution in previous layers to rotate weights rows
if (InferenceEngine::CNNNetHasPrevLayer(l.get())) {
std::vector<TranspositionInfo> transpositionInfo;
auto prevLayer = InferenceEngine::CNNNetPrevLayer(l);
transpositionInfo = FindTranspositionInfoFromPrevLayers(prevLayer);
if (FoundPartToTranspose(transpositionInfo)) {
if (l->input()->getDims().front() > 1) {
THROW_GNA_EXCEPTION
<< l->name << " Weights transposition is not supported for a layer with batch size > 1";
}
if (LayerInfo(prevLayer).isSplit()) {
// If we found a split it's not possible to rotate data
THROW_GNA_EXCEPTION << l->name << " won't be transposed due to a split before it";
}
transpInfoMatchWeightsSize(transpositionInfo, weightsColumns, l->name);
weightable->_weights = convertToRWBlob(weightable->_weights);
ConvertTensorFromNCHWToNHWC(precision,
weightsRows,
weightsColumns,
weightable->_weights->buffer().as<uint8_t*>(),
true,
transpositionInfo);
log::debug() << l->name << " weights rows transposition info:\n";
printTranspositionInfo(transpositionInfo);
}
}
// Find a convolution in next layers to rotate weights columns
if (!l->outData.empty() && !getInputTo(l->outData[0]).empty()) {
std::vector<TranspositionInfo> transpositionInfo;
auto nextLayer = getInputTo(l->outData[0]).begin()->second;
transpositionInfo = FindTranspositionInfoFromNextLayers(nextLayer);
if (FoundPartToTranspose(transpositionInfo)) {
if (l->outData[0]->getDims().front() > 1) {
THROW_GNA_EXCEPTION
<< l->name << " Weights transposition is not supported for a layer with batch size > 1";
}
if (LayerInfo(nextLayer).isConcat()) {
// If we found a concat it's not possible to rotate data
THROW_GNA_EXCEPTION << l->name << " won't be transposed due to a concat after it";
}
transpInfoMatchWeightsSize(transpositionInfo, weightsRows, l->name);
weightable->_weights = convertToRWBlob(weightable->_weights);
ConvertTensorFromNCHWToNHWC(precision,
weightsRows,
weightsColumns,
weightable->_weights->cbuffer().as<uint8_t*>(),
false,
transpositionInfo);
log::debug() << l->name << " weights columns transposition info:\n";
printTranspositionInfo(transpositionInfo);
}
}
}
if (LayerInfo(l).isEltwise()) {
// We need to transpose a constant which is an eltwise input
auto firstInput = InferenceEngine::CNNNetPrevLayer(l, 0);
auto secondInput = InferenceEngine::CNNNetPrevLayer(l, 1);
if (!LayerInfo(firstInput).isConst() && !LayerInfo(secondInput).isConst()) {
continue;
}
// Let a constant to be the second input
if (LayerInfo(firstInput).isConst()) {
std::swap(firstInput, secondInput);
}
// Find a convolution in previous or next layers
auto transpositionInfo = FindTranspositionInfoFromPrevLayers(firstInput);
if (!FoundPartToTranspose(transpositionInfo) && !l->outData.empty() && !getInputTo(l->outData[0]).empty()) {
transpositionInfo = FindTranspositionInfoFromNextLayers(getInputTo(l->outData[0]).begin()->second);
}
if (FoundPartToTranspose(transpositionInfo)) {
auto blob = secondInput->blobs["custom"];
ConvertTensorFromNCHWToNHWC(blob->getTensorDesc().getPrecision().size(),
1,
blob->size(),
blob->buffer().as<uint8_t*>(),
true,
transpositionInfo);
log::debug() << secondInput->name << " data transposition info:\n";
printTranspositionInfo(transpositionInfo);
}
}
if (LayerInfo(l).isConcat()) {
auto concatLayer = LayerInfo(l).as<InferenceEngine::ConcatLayer*>();
IE_ASSERT(concatLayer != nullptr);
// If concatenation is along channel axis constant input transposition isn't required
if (concatLayer->_axis <= 1)
continue;
std::vector<InferenceEngine::CNNLayerPtr> constInputs;
bool transpose = false;
int nonConstInputIx = 0;
// Check if non-const inputs are transposed
for (int i = 0; InferenceEngine::CNNNetHasPrevLayer(l.get(), i); ++i) {
auto input = InferenceEngine::CNNNetPrevLayer(l, i);
if (LayerInfo(input).isConst()) {
constInputs.push_back(input);
continue;
}
auto transpositionInfo = FindTranspositionInfoFromPrevLayers(input);
bool transposeInput = FoundPartToTranspose(transpositionInfo);
if (nonConstInputIx == 0) {
transpose = transposeInput;
} else if (transposeInput != transpose) {
THROW_GNA_EXCEPTION << "Concat layer " << l->name << " inputs have different layouts";
}
++nonConstInputIx;
}
if (!transpose)
continue;
// Transpose all constant inputs
for (auto&& input : constInputs) {
auto rows = GetDataDimByName(input->outData[0], DataDimName::C);
auto columns = GetDataDimByName(input->outData[0], DataDimName::H) *
GetDataDimByName(input->outData[0], DataDimName::W);
auto blob = convertToRWBlob(input->blobs["custom"]);
input->blobs["custom"] = blob;
// A constant should have the same number of channels since concatenation will be in height/weight
// dimension
TranspositionInfo concatTranspositionInfo{true, rows, columns};
ConvertTensorFromNCHWToNHWC(blob->getTensorDesc().getPrecision().size(),
1,
blob->size(),
blob->buffer().as<uint8_t*>(),
true,
{concatTranspositionInfo});
log::debug() << input->name << " data transposition info:\n";
printTranspositionInfo({concatTranspositionInfo});
}
}
}
}
void FuseFullyConnectedWithEltwisePass::run() {
// This legacy pass removes the Eltwise (only if it performs SUM op) from between FC and Any.
// The blob data of Const layer attached to Eltwise is added to biases blob data of FC layer.

View File

@@ -214,14 +214,6 @@ DECL_PASS(FuseFQIntoWeights);
*/
DECL_PASS(MoveFakeQuantizeLayerIntoQuantParams);
/**
* @brief convert FullyConnected, ScaleShift and Eltwise layers weights order from NCHW to NHWC.
* Information for transposition is found from convolution/pooling input or output dimensions.
* Convolution weights are transposed in finalizeConvolution1DPrimitive() method (gna_graph_compiler.cpp).
* They are transposed for the both, NCHW and NHWC models since MO always stores them in NCHW layout.
*/
DECL_PASS(TransposeWeightsFromNCHWToNHWC);
/**
* @brief fuse FullyConnected and Eltwise layers, also in case there is a Reshape between them having input with only
* one dimension > 1

View File

@@ -26,7 +26,7 @@ struct MVNData {
size_t W;
size_t num_parts;
float eps;
op::MVNEpsMode eps_mode;
ov::op::MVNEpsMode eps_mode;
bool normalize_variance;
element::Type element_type;
std::string name;
@@ -138,7 +138,7 @@ static std::shared_ptr<Node> NormalizeVariance(const std::shared_ptr<opset8::MVN
CoordinateDiff{0, 0},
CoordinateDiff{0, 0},
Strides{1, 1},
op::PadType::VALID);
ov::op::PadType::VALID);
transposed_avg_conv_3->set_friendly_name(mvn_data.name + "_Avg3");
auto avg_conv_3 =
std::make_shared<opset8::Transpose>(transposed_avg_conv_3,
@@ -156,7 +156,7 @@ static std::shared_ptr<Node> NormalizeVariance(const std::shared_ptr<opset8::MVN
CoordinateDiff{0, 0},
CoordinateDiff{0, 0},
Strides{1, 1},
op::PadType::VALID);
ov::op::PadType::VALID);
transposed_avg_conv_4->set_friendly_name(mvn_data.name + "_Avg4");
auto avg_conv_4 =
std::make_shared<opset8::Transpose>(transposed_avg_conv_4,
@@ -243,7 +243,7 @@ static void Decompose(const std::shared_ptr<opset8::MVN> mvn, const MVNData& mvn
CoordinateDiff{0, 0},
CoordinateDiff{0, 0},
Strides{1, 1},
op::PadType::VALID);
ov::op::PadType::VALID);
transposed_avg_conv_1->set_friendly_name(mvn_data.name + "_Avg1");
auto avg_conv_1 =
std::make_shared<opset8::Transpose>(transposed_avg_conv_1,
@@ -261,7 +261,7 @@ static void Decompose(const std::shared_ptr<opset8::MVN> mvn, const MVNData& mvn
CoordinateDiff{0, 0},
CoordinateDiff{0, 0},
Strides{1, 1},
op::PadType::VALID);
ov::op::PadType::VALID);
transposed_avg_conv_2->set_friendly_name(mvn_data.name + "_Avg2");
auto avg_conv_2 =
std::make_shared<opset8::Transpose>(transposed_avg_conv_2,

View File

@@ -1,4 +1,4 @@
// Copyright (C) 2018-2023 Intel Corporation
// Copyright (C) 2023 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#include "fuse_conv_bias_activation.hpp"

View File

@@ -12,6 +12,9 @@
#include "openvino/pass/pattern/op/wrap_type.hpp"
#include "transformations/gather_sinking_binary.hpp"
#include "transformations/gather_sinking_fuse.hpp"
#include "transformations/gather_sinking_matmul.hpp"
#include "transformations/gather_sinking_reshape.hpp"
#include "transformations/gather_sinking_split.hpp"
#include "transformations/gather_sinking_transpose_reshape.hpp"
#include "transformations/gather_sinking_unary.hpp"
@@ -25,6 +28,7 @@ GatherSinkingGeneralForward::GatherSinkingGeneralForward() {
add_matcher<GatherSinkingUnaryForward>();
add_matcher<GatherSinkingBinaryForward>();
add_matcher<GatherSinkingTransposeReshapeForward>();
add_matcher<GatherSinkingMatmulForward>();
add_matcher<GatherSinkingFuse>();
}
@@ -33,6 +37,9 @@ GatherSinkingGeneralBackward::GatherSinkingGeneralBackward() {
add_matcher<GatherSinkingUnaryBackward>();
add_matcher<GatherSinkingBinaryBackward>();
add_matcher<GatherSinkingTransposeReshapeBackward>();
add_matcher<GatherSinkingReshapeBackward>();
add_matcher<GatherSinkingSplitBackward>();
add_matcher<GatherSinkingMatmulBackward>();
add_matcher<GatherSinkingFuse>();
}

View File

@@ -14,9 +14,11 @@
#include <openvino/cc/ngraph/itt.hpp>
#include "backend/gna_limitations.hpp"
#include "common/graph_utils.hpp"
using namespace ov::intel_gna::pass;
using namespace ov::intel_gna::limitations;
using namespace ov::intel_gna::graph_utils;
namespace {
@@ -171,7 +173,7 @@ HandleTransposeBeforeMatMul::HandleTransposeBeforeMatMul() {
auto iter = pattern_map.find(fq);
if (iter != pattern_map.end() || (iter = pattern_map.find(constant)) != pattern_map.end()) {
auto prev_node = iter->second.get_node_shared_ptr();
if (Limitations::is_transpose_2d(prev_node->get_output_shape(0))) {
if (is_shape_2d(prev_node->get_output_shape(0))) {
InsertTranspose(prev_node, prev_node->get_friendly_name(), true);
}
}

View File

@@ -34,7 +34,7 @@ bool MarkupFusableTranspose::run_on_model(const std::shared_ptr<ngraph::Function
for (auto& node : f->get_ordered_ops()) {
if (!std::dynamic_pointer_cast<ngraph::opset9::Convolution>(node) &&
!std::dynamic_pointer_cast<ngraph::op::ConvolutionIE>(node)) {
!std::dynamic_pointer_cast<ov::intel_gna::op::GNAConvolution>(node)) {
continue;
}
auto in_dims = node->input(0).get_shape();

View File

@@ -47,7 +47,7 @@ ov::Shape make_transpose_order_nchw2nhwc(size_t shape_size) {
ov::Shape shape(shape_size);
std::iota(shape.begin(), shape.end(), 0);
for (int i = 1; i < shape.size() - 1; ++i)
for (size_t i = 1; i < shape.size() - 1; ++i)
shape[i] = shape[i + 1];
*(shape.end() - 1) = 1;

View File

@@ -0,0 +1,162 @@
// Copyright (C) 2023 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#include "transformations/reshape_transpose_substitute.hpp"
#include <openvino/cc/ngraph/itt.hpp>
#include <transformations/utils/utils.hpp>
#include <utility>
#include "openvino/opsets/opset12.hpp"
#include "openvino/pass/pattern/op/wrap_type.hpp"
using namespace ov;
using namespace ov::opset12;
using namespace ov::pass::pattern;
using namespace ov::op::util;
using namespace ov::intel_gna::pass;
namespace {
/**
* @brief Checks if output_shape is just permutation of input_shape dimensions
*/
bool is_shape_permutation(const Shape& input_shape, const Shape& output_shape) {
std::unordered_map<Shape::value_type, size_t> values_count;
for (const auto& dim : input_shape) {
auto it = values_count.find(dim);
if (it != values_count.end()) {
++it->second;
continue;
}
values_count[dim] = 1;
}
for (const auto& dim : output_shape) {
auto it = values_count.find(dim);
if (it != values_count.end()) {
--it->second;
if (!it->second) {
values_count.erase(it);
}
continue;
}
return false;
}
return values_count.empty();
}
/**
* @brief Checks if all shape dimensions have different values
*/
bool are_dims_unique(const Shape& shape) {
std::unordered_set<Shape::value_type> dims;
for (const auto& dim : shape) {
if (dims.find(dim) != dims.end())
return false;
dims.insert(dim);
}
return true;
}
AxisVector get_unique_shapes_transpose_order(const Shape& input_shape, const Shape& output_shape) {
std::unordered_map<Shape::value_type, size_t> input_shape_items;
for (size_t i = 0; i < input_shape.size(); ++i) {
input_shape_items[input_shape[i]] = i;
}
AxisVector order;
order.reserve(output_shape.size());
for (const auto& output_dim : output_shape) {
order.push_back(input_shape_items.find(output_dim)->second);
}
return order;
}
/**
* @brief Changes shape according to specified order
*/
Shape apply_permutation(const Shape& shape, const AxisVector& order) {
Shape transposed_shape;
transposed_shape.reserve(shape.size());
for (const auto& position : order) {
transposed_shape.push_back(shape[position]);
}
return transposed_shape;
}
AxisVector find_suitable_transpose_order(const Shape& input_shape,
const Shape& output_shape,
const std::vector<AxisVector>& orders) {
for (const auto& order : orders) {
const Shape transposed_shape = apply_permutation(input_shape, order);
if ((transposed_shape.size() == output_shape.size()) &&
std::equal(transposed_shape.begin(), transposed_shape.end(), output_shape.begin()))
return order;
}
return {};
}
AxisVector find_suitable_transpose_order(const Shape& input_shape, const Shape& output_shape) {
static std::vector<AxisVector> orders_4d = {AxisVector{0, 2, 3, 1}, AxisVector{0, 3, 1, 2}};
static std::vector<AxisVector> orders_3d = {AxisVector{1, 2, 0}, AxisVector{2, 0, 1}};
switch (input_shape.size()) {
case 4:
return find_suitable_transpose_order(input_shape, output_shape, orders_4d);
case 3:
return find_suitable_transpose_order(input_shape, output_shape, orders_3d);
case 2:
return AxisVector{1, 0};
default:
return {};
}
return {};
}
/**
* @brief Finds applicable transpose order to get output_shape from input_shape if
* it is possible
* Returns empty AxisVector on unsupported case
*/
AxisVector get_transpose_order(const Shape& input_shape, const Shape& output_shape) {
if (are_dims_unique(input_shape))
return get_unique_shapes_transpose_order(input_shape, output_shape);
return find_suitable_transpose_order(input_shape, output_shape);
}
} // namespace
ReshapeTransposeSubstitute::ReshapeTransposeSubstitute() {
MATCHER_SCOPE(ReshapeTransposeSubstitute);
auto reshape_label = wrap_type<Reshape>({any_input(), any_input()}, [](const Output<Node>& output) {
if (!has_static_shape()(output))
return false;
const Node* node = output.get_node();
return is_shape_permutation(node->get_input_shape(0), node->get_output_shape(0));
});
ov::matcher_pass_callback matcher_pass_callback = [=](Matcher& m) {
const auto& pattern_to_output = m.get_pattern_value_map();
auto reshape = as_type_ptr<Reshape>(pattern_to_output.at(reshape_label).get_node_shared_ptr());
const AxisVector transpose_order =
get_transpose_order(reshape->get_input_shape(0), reshape->get_output_shape(0));
if (transpose_order.empty())
return false;
auto new_transpose_const =
std::make_shared<Constant>(element::i64, Shape{transpose_order.size()}, transpose_order);
auto new_transpose = std::make_shared<Transpose>(reshape->input_value(0), new_transpose_const);
replace_node_update_name(reshape, new_transpose);
return true;
};
auto m = std::make_shared<Matcher>(reshape_label, matcher_name);
register_matcher(m, matcher_pass_callback);
}

View File

@@ -0,0 +1,34 @@
// Copyright (C) 2023 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#pragma once
#include "openvino/pass/graph_rewrite.hpp"
#include "transformations_visibility.hpp"
namespace ov {
namespace intel_gna {
namespace pass {
/**
* @brief Substitute Reshape with Transpose if it possible.
* It is usefull with Tranpose/Gather sinking transformations to simplify Transpose movement.
* TransposeSinking is not possible to use with Reshapes (except some cases when Reshape is Squeeze/Unsqueeze).
* GatherSinking is much harder to implement for all existed layer types.
*
* any layer any layer
* | |
* Reshape => Transpose
* | |
* any layer any layer
*/
class ReshapeTransposeSubstitute : public ov::pass::MatcherPass {
public:
OPENVINO_RTTI("ReshapeTransposeSubstitute", "0");
ReshapeTransposeSubstitute();
};
} // namespace pass
} // namespace intel_gna
} // namespace ov

View File

@@ -0,0 +1,79 @@
// Copyright (C) 2023 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#include "transformations/rotate_inputs.hpp"
#include "common/graph_utils.hpp"
#include "openvino/cc/ngraph/itt.hpp"
#include "openvino/opsets/opset11.hpp"
#include "openvino/pass/pattern/op/wrap_type.hpp"
#include "ops/gna_convolution.hpp"
using namespace ov::opset11;
using namespace ov::pass;
using namespace ov::intel_gna::pass;
using namespace ov::intel_gna::graph_utils;
namespace {
inline bool is_skip_operation(const std::shared_ptr<ov::Node>& node) {
return (std::dynamic_pointer_cast<Reshape>(node) != nullptr ||
std::dynamic_pointer_cast<Transpose>(node) != nullptr ||
std::dynamic_pointer_cast<Squeeze>(node) != nullptr ||
std::dynamic_pointer_cast<Unsqueeze>(node) != nullptr ||
std::dynamic_pointer_cast<FakeQuantize>(node) != nullptr) &&
has_n_consumers(node, 1);
}
} // namespace
InsertConvolutionTransposeHW::InsertConvolutionTransposeHW() {
MATCHER_SCOPE(InsertConvolutionTransposeHW);
const auto conv_pattern = pattern::wrap_type<op::GNAConvolution>(
{pattern::any_input(), pattern::any_input()},
[](const ov::Output<ov::Node>& node) {
std::shared_ptr<op::GNAConvolution> conv =
std::dynamic_pointer_cast<op::GNAConvolution>(node.get_node_shared_ptr());
helper::ConvData conv_data;
helper::GetConvData(conv, conv_data);
return gna_convolution_layer::should_transpose_h_w(static_cast<uint32_t>(conv_data.input_height),
static_cast<uint32_t>(conv_data.filter_height),
static_cast<uint32_t>(conv_data.input_channel_count),
static_cast<uint32_t>(conv_data.filter_stride_height));
});
ov::matcher_pass_callback callback = [=](pattern::Matcher& m) {
const auto& pattern_map = m.get_pattern_value_map();
// auto param_node = pattern_map.at(param_pattern).get_node_shared_ptr();
auto conv_node = pattern_map.at(conv_pattern).get_node_shared_ptr();
std::shared_ptr<ov::Node> target_node =
graph_utils::get_prev_node_skipping_certain(conv_node->get_input_node_shared_ptr(0), is_skip_operation);
std::shared_ptr<Parameter> param_node = std::dynamic_pointer_cast<Parameter>(target_node);
if (!param_node) {
return false;
}
// transpose all convolution inputs
for (const auto& conv_input : conv_node->inputs()) {
// Transpose H and W (NHWC -> NWHC)
ov::AxisVector tr_axis = {0, 2, 1, 3};
auto transpose_const = std::make_shared<Constant>(ov::element::i8, ov::Shape{tr_axis.size()}, tr_axis);
auto transpose = std::make_shared<Transpose>(conv_input.get_source_output(), transpose_const);
// Reshape out
ov::Shape shape_out = conv_input.get_shape();
auto reshape_out_const =
std::make_shared<Constant>(ov::element::i32, ov::Shape{shape_out.size()}, shape_out);
auto reshape_out = std::make_shared<Reshape>(transpose, reshape_out_const, false);
conv_input.replace_source_output(reshape_out);
}
return true;
};
auto m = std::make_shared<pattern::Matcher>(conv_pattern, matcher_name);
this->register_matcher(m, callback);
}

View File

@@ -0,0 +1,41 @@
// Copyright (C) 2023 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#pragma once
#include "openvino/pass/graph_rewrite.hpp"
namespace ov {
namespace intel_gna {
namespace pass {
/**
* @brief Transpose convolution inputs (HW->WH)
* when it is directly connected to network inputs
*
* Searches for next pattern
* Any input layer
* |
* Reshape/FQ/Squeeze/Usqueeze
* |
* Convolution
*
* And transforms to
* Any input layer
* |
* Transpose
* |
* Reshape/FQ/Squeeze/Usqueeze
* |
* Convolution
*/
class InsertConvolutionTransposeHW : public ov::pass::MatcherPass {
public:
OPENVINO_RTTI("InsertConvolutionTransposeHW", "0");
InsertConvolutionTransposeHW();
};
} // namespace pass
} // namespace intel_gna
} // namespace ov

View File

@@ -2,7 +2,7 @@
// SPDX-License-Identifier: Apache-2.0
//
#include "transformations/ts_concat.hpp"
#include "transformations/ts_concat_forward.hpp"
#include <openvino/cc/ngraph/itt.hpp>
@@ -23,7 +23,6 @@ using namespace ov::intel_gna::pass::helper;
using namespace ov::intel_gna::limitations;
namespace {
bool is_concat_sinked(const Output<Node>& output) {
auto concat_node = ov::as_type_ptr<Concat>(output.get_node_shared_ptr());
@@ -41,7 +40,6 @@ bool is_concat_sinked(const Output<Node>& output) {
return false;
}
} // namespace
TSConcatForward::TSConcatForward() {

View File

@@ -2,7 +2,7 @@
// SPDX-License-Identifier: Apache-2.0
//
#include "transformations/ts_split.hpp"
#include "transformations/ts_split_backward.hpp"
#include <openvino/cc/ngraph/itt.hpp>

View File

@@ -6,17 +6,20 @@
#include "log/debug.hpp"
#include "openvino/core/rt_info.hpp"
#include "openvino/opsets/opset10.hpp"
#include "openvino/opsets/opset12.hpp"
#include "openvino/pass/pattern/op/wrap_type.hpp"
#include "ops/gna_convolution.hpp"
#include "ops/gna_max_pool.hpp"
#include "transformations/rt_info/transpose_sinking_attr.hpp"
using namespace ov::opset12;
namespace ov {
namespace intel_gna {
namespace pass {
namespace helper {
using namespace ov::opset10;
void GetConvData(std::shared_ptr<ngraph::op::ConvolutionIE> conv, ConvData& conv_data) {
void GetConvData(std::shared_ptr<ngraph::opset7::Convolution> conv, ConvData& conv_data) {
OPENVINO_ASSERT(conv);
conv_data.output_height = conv->get_output_shape(0)[2];
conv_data.output_width = conv->get_output_shape(0)[3];
@@ -40,17 +43,17 @@ void GetConvData(std::shared_ptr<ngraph::op::ConvolutionIE> conv, ConvData& conv
conv_data.element_type = conv->get_element_type();
}
void GetConvData(std::shared_ptr<Convolution> conv, ConvData& conv_data) {
void GetConvData(std::shared_ptr<ov::intel_gna::op::GNAConvolution> conv, ConvData& conv_data) {
OPENVINO_ASSERT(conv);
conv_data.output_height = conv->get_output_shape(0)[2];
conv_data.output_width = conv->get_output_shape(0)[3];
conv_data.input_channel_count = conv->input_value(0).get_shape()[1];
conv_data.input_height = conv->input_value(0).get_shape()[2];
conv_data.input_width = conv->input_value(0).get_shape()[3];
conv_data.input_channel_count = conv->input_value(0).get_shape()[3];
conv_data.input_height = conv->input_value(0).get_shape()[1];
conv_data.input_width = conv->input_value(0).get_shape()[2];
conv_data.filter_count = conv->input_value(1).get_shape()[0];
conv_data.filter_channel_count = conv->input_value(1).get_shape()[1];
conv_data.filter_height = conv->input_value(1).get_shape()[2];
conv_data.filter_width = conv->input_value(1).get_shape()[3];
conv_data.filter_channel_count = conv->input_value(1).get_shape()[3];
conv_data.filter_height = conv->input_value(1).get_shape()[1];
conv_data.filter_width = conv->input_value(1).get_shape()[2];
conv_data.filter_dilation_height = conv->get_dilations()[0];
conv_data.filter_dilation_width = conv->get_dilations()[1];
conv_data.filter_stride_height = conv->get_strides()[0];
@@ -180,6 +183,75 @@ void swap_names(std::shared_ptr<ov::Node> node1, std::shared_ptr<ov::Node> node2
swap_output_names(node1->output(0), node2->output(0));
}
ov::AxisVector reverse_transpose_order(const ov::AxisVector& axis_order) {
ov::AxisVector out(axis_order.size());
for (size_t i = 0; i < axis_order.size(); i++) {
out.at(axis_order[i]) = i;
}
return out;
}
ov::NodeVector find_input_transposes(const std::shared_ptr<const ov::Node>& node) {
ov::NodeVector transposes;
for (size_t input_idx = 0; input_idx < node->get_input_size(); ++input_idx) {
auto input_node = node->get_input_node_shared_ptr(input_idx);
auto transpose_node = ov::as_type_ptr<Transpose>(input_node);
if (transpose_node)
transposes.push_back(transpose_node);
}
return transposes;
}
void mark_input_transposes_as_nosinking(std::shared_ptr<const ov::Node> node) {
for (const auto& input : find_input_transposes(node))
ov::mark_as_no_sinking_node(input);
}
TransposeInfo get_first_input_transpose(const std::shared_ptr<const ov::Node>& node) {
for (size_t input_idx = 0; input_idx < node->get_input_size(); ++input_idx) {
std::shared_ptr<Node> input_node = node->get_input_node_shared_ptr(input_idx);
auto transpose_node = as_type_ptr<Transpose>(input_node);
if (!transpose_node)
continue;
auto constant_node = as_type_ptr<Constant>(transpose_node->input_value(1).get_node_shared_ptr());
if (!constant_node)
continue;
{
TransposeInfo input_info;
input_info.transpose = transpose_node;
input_info.transpose_const = constant_node;
return input_info;
}
}
return {};
}
TransposeInfo get_first_output_transpose(const std::shared_ptr<const ov::Node>& node) {
for (size_t i = 0; i < node->get_output_size(); ++i) {
for (const auto& input : node->output(i).get_target_inputs()) {
Node* node = input.get_node();
if (!dynamic_cast<Transpose*>(node))
continue;
auto transpose_node = as_type_ptr<Transpose>(node->shared_from_this());
if (!transpose_node)
continue;
auto constant_node = as_type_ptr<Constant>(transpose_node->input_value(1).get_node_shared_ptr());
if (!constant_node)
continue;
{
TransposeInfo input_info;
input_info.transpose = transpose_node;
input_info.transpose_const = constant_node;
return input_info;
}
}
}
return {};
}
} // namespace helper
} // namespace pass
} // namespace intel_gna

View File

@@ -7,6 +7,9 @@
#include <legacy/ngraph_ops/convolution_ie.hpp>
#include <ngraph/opsets/opset7.hpp>
#include "openvino/opsets/opset12.hpp"
#include "ops/gna_convolution.hpp"
namespace ov {
namespace intel_gna {
namespace pass {
@@ -44,12 +47,12 @@ struct ConvData {
void GetConvData(std::shared_ptr<ngraph::opset7::Convolution> conv, ConvData& conv_data);
/**
* @brief gets all legacy convolution related data into a struct for further processing
* @param conv legacy convolution node to get data of
* @brief gets all convolution related data into a struct for further processing
* @param conv GNA custom convolution node to get data of
* @param conv_data convolution data structure to put data into
* @return void
*/
void GetConvData(std::shared_ptr<ngraph::op::ConvolutionIE> conv, ConvData& conv_data);
void GetConvData(std::shared_ptr<ov::intel_gna::op::GNAConvolution> conv, ConvData& conv_data);
/**
* @brief ngraph matcher predicate fusing existing predicates for consumers count and rank of a layer
@@ -116,6 +119,41 @@ void swap_friendly_names(std::shared_ptr<ov::Node>, std::shared_ptr<ov::Node>);
*/
void swap_names(std::shared_ptr<ov::Node>, std::shared_ptr<ov::Node>);
/**
* @brief Reverses axis order. Final result will be such an order, that together
* with initial order will be {0, 1, 2, ...}
*/
ov::AxisVector reverse_transpose_order(const ov::AxisVector& axis_order);
/**
* @brief Finds all input node transposes
*/
ov::NodeVector find_input_transposes(const std::shared_ptr<const ov::Node>& node);
/**
* @brief Marks all input transposes with flag NoSinking
*/
void mark_input_transposes_as_nosinking(std::shared_ptr<const ov::Node> node);
struct TransposeInfo {
std::shared_ptr<ov::opset12::Transpose> transpose;
std::shared_ptr<ov::opset12::Constant> transpose_const;
bool isEmpty() const {
return !transpose || !transpose_const;
}
};
/**
* @brief Finds first input node transpose
*/
TransposeInfo get_first_input_transpose(const std::shared_ptr<const ov::Node>& node);
/**
* @brief Finds first output node transpose
*/
TransposeInfo get_first_output_transpose(const std::shared_ptr<const ov::Node>& node);
} // namespace helper
} // namespace pass
} // namespace intel_gna

View File

@@ -273,32 +273,6 @@ TEST_F(I16QuantisationTest, EltwiseSumm_onlyOneIdentityInsertion) {
.once();
}
TEST_F(I16QuantisationTest, canDetectLeakyRelu) {
assert_that()
.onInferModel(TFLeakyReluModel())
.inNotCompactMode()
.withGNAConfig(GNA_CONFIG_KEY(SCALE_FACTOR), 1.0f)
.gna()
.propagate_forward()
.called_with()
.pwl_inserted_into_nnet();
}
TEST_F(I16QuantisationTest, MaxPool_followedAfterActivation) {
assert_that()
.onInferModel(maxpoolAfterRelu())
.inNotCompactMode()
.withGNAConfig(GNA_CONFIG_KEY(SCALE_FACTOR), 1.0f)
.gna()
.propagate_forward()
.called_with()
.convolution_inserted_into_nnet()
.And()
.pwl_inserted_into_nnet()
.And()
.max_pooling_inserted_into_nnet();
}
TEST_F(I16QuantisationTest, EltwiseMull_willInsertTwoIdentities) {
assert_that()
.onInferModel(eltwiseMulModel())

View File

@@ -0,0 +1,137 @@
// Copyright (C) 2023 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#include <map>
#include <numeric>
#include <ostream>
#include <string>
#include <vector>
#include "common_test_utils/common_utils.hpp"
#include "ngraph_functions/builders.hpp"
#include "openvino/opsets/opset12.hpp"
#include "shared_test_classes/base/layer_test_utils.hpp"
#include "shared_test_classes/base/ov_subgraph.hpp"
using namespace ov::opset12;
using namespace ov::test;
namespace TransposesConcatTests {
typedef std::tuple<ov::element::Type, // Net precision
std::string, // Device name
std::map<std::string, std::string>, // Configuration
std::map<std::string, std::string> // Additional Configuration
>
TransposesConcatTestParamsSet;
class TransposesConcatTest : public testing::WithParamInterface<TransposesConcatTestParamsSet>,
virtual public SubgraphBaseTest {
public:
static std::string get_test_case_name(const testing::TestParamInfo<TransposesConcatTestParamsSet>& obj) {
ov::element::Type net_type, in_type, out_type;
std::string target_device;
std::map<std::string, std::string> conf, conf_ext;
std::tie(net_type, target_device, conf, conf_ext) = obj.param;
for (auto& conf_item : conf_ext) {
conf[conf_item.first] = conf_item.second;
}
std::ostringstream result;
result << "netPRC=" << net_type << "_";
result << "trgDev=" << target_device;
for (auto const& conf_i : conf) {
result << "_configItem=" << conf_i.first.c_str() << "_" << conf_i.second.c_str();
}
return result.str();
}
protected:
void SetUp() override {
abs_threshold = std::numeric_limits<int32_t>::max();
rel_threshold = std::numeric_limits<int32_t>::max();
std::map<std::string, std::string> conf, conf_ext;
std::tie(m_net_type, targetDevice, conf, conf_ext) = this->GetParam();
std::vector<InputShape> input_shapes = static_shapes_to_test_representation({{10, 1}});
configuration.insert(conf.begin(), conf.end());
for (auto& conf_item : conf_ext) {
configuration[conf_item.first] = conf_item.second;
}
init_input_shapes(input_shapes);
}
void init_test_model() {
std::vector<std::vector<size_t>> input_shapes = {{10, 1}};
auto params = ngraph::builder::makeParams(m_net_type, input_shapes);
std::vector<size_t> shape_1 = {10, 128};
std::vector<size_t> shape_2 = {10, 192};
std::vector<size_t> shape_3 = {10, 256};
std::shared_ptr<ov::Node> input_node = params[0];
std::vector<float> mulConstWeights(1 * 576);
std::iota(mulConstWeights.begin(), mulConstWeights.end(), 0.1f);
auto constMul1 = ngraph::builder::makeConstant<float>(m_net_type, ov::Shape{1, 576}, mulConstWeights);
auto matmul1 = std::make_shared<ov::opset10::MatMul>(input_node, constMul1, false, false);
auto split_axis = std::make_shared<Constant>(ov::element::u8, ov::Shape{1}, std::vector<uint8_t>{1});
auto split_slices =
std::make_shared<Constant>(ov::element::u32, ov::Shape{3}, std::vector<uint32_t>{128, 192, 256});
auto split_node = std::make_shared<VariadicSplit>(matmul1, split_axis, split_slices);
std::vector<size_t> transpose_order = {1, 0};
auto transpose_const_1 =
std::make_shared<Constant>(ov::element::u8, ov::Shape{transpose_order.size()}, transpose_order);
auto transpose_const_2 =
std::make_shared<Constant>(ov::element::u8, ov::Shape{transpose_order.size()}, transpose_order);
auto transpose_const_3 =
std::make_shared<Constant>(ov::element::u8, ov::Shape{transpose_order.size()}, transpose_order);
auto transpose_node_1 = std::make_shared<Transpose>(split_node->output(0), transpose_const_1);
auto transpose_node_2 = std::make_shared<Transpose>(split_node->output(1), transpose_const_2);
auto transpose_node_3 = std::make_shared<Transpose>(split_node->output(2), transpose_const_3);
const int axis = 0;
auto concat_node =
std::make_shared<Concat>(ov::OutputVector{transpose_node_1, transpose_node_2, transpose_node_3}, axis);
std::vector<size_t> reshape_pattern = {1, 2, 5, 576};
auto reshape_const =
std::make_shared<Constant>(ov::element::u16, ov::Shape{reshape_pattern.size()}, reshape_pattern);
auto reshape_node = std::make_shared<Reshape>(concat_node, reshape_const, false);
ov::ResultVector results{std::make_shared<Result>(reshape_node)};
function = std::make_shared<ov::Model>(results, params, "concat");
}
ov::element::Type m_net_type;
std::vector<size_t> m_input_shape;
};
TEST_P(TransposesConcatTest, CompareWithRefs) {
init_test_model();
run();
}
std::vector<std::map<std::string, std::string>> configs = {{{"GNA_DEVICE_MODE", "GNA_SW_EXACT"}}};
std::vector<std::map<std::string, std::string>> target_configs = {{{"GNA_DEVICE_MODE", "GNA_SW_FP32"}},
{{"GNA_EXEC_TARGET", "GNA_TARGET_2_0"}},
{{"GNA_EXEC_TARGET", "GNA_TARGET_3_0"}},
{{"GNA_EXEC_TARGET", "GNA_TARGET_3_5"}}};
const ov::element::TypeVector input_precisions = {ov::element::f32};
INSTANTIATE_TEST_SUITE_P(smoke_transposes_concat,
TransposesConcatTest,
::testing::Combine(::testing::ValuesIn(input_precisions),
::testing::Values(CommonTestUtils::DEVICE_GNA),
::testing::ValuesIn(configs),
::testing::ValuesIn(target_configs)),
TransposesConcatTest::get_test_case_name);
} // namespace TransposesConcatTests

View File

@@ -108,7 +108,7 @@ void ConvolutionLayerTestFixture::SetUp() {
auto filter_size = std::accumulate(std::begin(kernel), std::end(kernel), 1, std::multiplies<size_t>());
filter_weights =
CommonTestUtils::generate_float_numbers(convOutChannels * inputShape[1] * filter_size, -0.5f, 0.5f);
CommonTestUtils::generate_float_numbers(convOutChannels * inputShape[1] * filter_size, -0.1f, 0.1f);
auto conv = std::dynamic_pointer_cast<Convolution>(ngraph::builder::makeConvolution(paramOuts[0],
ngPrc,

View File

@@ -310,11 +310,11 @@ class GNAMemoryOrderTest : public ::testing::Test {};
TEST_F(GNAMemoryOrderTest, orderingFusedLayersActivation) {
auto plugin = GNAPluginTested();
ov::Shape input_shape = {1, 8, 20, 16};
ov::Shape input_shape = {1, 16, 20, 16};
ov::Strides strides = {1, 1};
ov::Strides dilations = {1, 1};
ov::CoordinateDiff pad_begin(0, 0), pad_end(0, 0);
auto weights = ngraph::builder::makeConstant<float>(ov::element::f32, {8, 8, 1, 1}, {1.f});
auto weights = ngraph::builder::makeConstant<float>(ov::element::f32, {8, 16, 1, 1}, {1.f});
auto input = std::make_shared<ngraph::opset8::Parameter>(ov::element::f32, input_shape);
auto conv = std::make_shared<ngraph::opset8::Convolution>(input, weights, strides, pad_begin, pad_end, dilations);
@@ -332,11 +332,11 @@ TEST_F(GNAMemoryOrderTest, orderingFusedLayersActivation) {
TEST_F(GNAMemoryOrderTest, orderingFusedLayersMaxPool) {
auto plugin = GNAPluginTested();
ov::Shape input_shape = {1, 8, 20, 16};
ov::Shape input_shape = {1, 16, 20, 16};
ov::Strides strides = {1, 1};
ov::Strides dilations = {1, 1};
ov::CoordinateDiff pad_begin(0, 0), pad_end(0, 0);
auto weights = ngraph::builder::makeConstant<float>(ov::element::f32, {8, 8, 1, 1}, {1.f});
auto weights = ngraph::builder::makeConstant<float>(ov::element::f32, {8, 16, 1, 1}, {1.f});
auto input = std::make_shared<ngraph::opset8::Parameter>(ov::element::f32, input_shape);
auto conv = std::make_shared<ngraph::opset8::Convolution>(input, weights, strides, pad_begin, pad_end, dilations);
@@ -361,11 +361,11 @@ TEST_F(GNAMemoryOrderTest, orderingFusedLayersMaxPool) {
TEST_F(GNAMemoryOrderTest, orderingFusedLayersActivationMaxPool) {
auto plugin = GNAPluginTested();
ov::Shape input_shape = {1, 8, 20, 16};
ov::Shape input_shape = {1, 16, 20, 16};
ov::Strides strides = {1, 1};
ov::Strides dilations = {1, 1};
ov::CoordinateDiff pad_begin(0, 0), pad_end(0, 0);
auto weights = ngraph::builder::makeConstant<float>(ov::element::f32, {8, 8, 1, 1}, {1.f});
auto weights = ngraph::builder::makeConstant<float>(ov::element::f32, {8, 16, 1, 1}, {1.f});
auto input = std::make_shared<ngraph::opset8::Parameter>(ov::element::f32, input_shape);
auto conv = std::make_shared<ngraph::opset8::Convolution>(input, weights, strides, pad_begin, pad_end, dilations);

View File

@@ -0,0 +1,130 @@
// Copyright (C) 2023 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#include "transformations/gather_sinking.hpp"
#include <ngraph/function.hpp>
#include <ngraph/pass/manager.hpp>
#include <openvino/opsets/opset10.hpp>
#include <ops/gna_convolution.hpp>
#include <ops/gna_max_pool.hpp>
#include <transformations/init_node_info.hpp>
#include "common_test_utils/ngraph_test_utils.hpp"
#include "gtest/gtest.h"
#include "transformations/gather_sinking_matmul.hpp"
#include "transformations/gather_sinking_reshape.hpp"
#include "transformations/gather_sinking_split.hpp"
#include "transformations/replace_gna_nhwc_layers.hpp"
using namespace ov;
using namespace ov::opset10;
TEST(TransposeNCHW, Convolution) {
std::shared_ptr<Model> function;
{
auto input_params = std::make_shared<Parameter>(element::Type_t::f32, Shape{1, 1, 41, 1});
auto kernel = Constant::create(ov::element::f32, {4, 1, 3, 1}, {1});
auto convolution = std::make_shared<Convolution>(input_params,
kernel,
Strides{2, 1},
CoordinateDiff{0, 0},
CoordinateDiff{0, 0},
Strides{1, 1});
const auto result = std::make_shared<Result>(convolution);
function = std::make_shared<Model>(OutputVector{result}, ParameterVector{input_params});
}
std::shared_ptr<Model> orig_function = function->clone();
ov::pass::Manager manager;
manager.register_pass<ov::pass::InitNodeInfo>();
manager.register_pass<ov::intel_gna::pass::SubstituteGNAConvolution>();
manager.run_passes(function);
ASSERT_NO_THROW(check_rt_info(function));
std::shared_ptr<Model> reference_function;
{
auto input_params = std::make_shared<Parameter>(element::Type_t::f32, Shape{1, 1, 41, 1});
auto transpose_before_const = Constant::create(element::i32, Shape{4}, {0, 2, 3, 1});
auto transpose_before = std::make_shared<Transpose>(input_params, transpose_before_const);
auto kernel = Constant::create(ov::element::f32, {4, 1, 3, 1}, {1});
auto transpose_conv_const = Constant::create(element::i32, Shape{4}, {0, 2, 3, 1});
auto transpose_conv_before = std::make_shared<Transpose>(input_params, transpose_conv_const);
auto transpose_conv_constant = std::make_shared<Transpose>(kernel, transpose_conv_const);
auto convolution = std::make_shared<ov::intel_gna::op::GNAConvolution>(transpose_before,
transpose_conv_constant,
Strides{2, 1},
CoordinateDiff{0, 0},
CoordinateDiff{0, 0},
Strides{1, 1});
auto transpose_after_const = Constant::create(element::i32, Shape{4}, {0, 3, 1, 2});
auto transpose_after = std::make_shared<Transpose>(convolution, transpose_after_const);
const auto result = std::make_shared<Result>(transpose_after);
reference_function = std::make_shared<Model>(OutputVector{result}, ParameterVector{input_params});
}
const FunctionsComparator func_comparator =
FunctionsComparator::with_default().enable(FunctionsComparator::ATTRIBUTES);
const FunctionsComparator::Result result = func_comparator(function, reference_function);
ASSERT_TRUE(result.valid) << result.message;
}
TEST(TransposeNCHW, MaxPool) {
std::shared_ptr<Model> function;
{
auto input_params = std::make_shared<Parameter>(element::Type_t::f32, Shape{1, 1, 41, 1});
auto max_pool =
std::make_shared<ov::op::v1::MaxPool>(input_params, Strides{2, 1}, Shape{0, 0}, Shape{0, 0}, Shape{4, 1});
const auto result = std::make_shared<Result>(max_pool);
function = std::make_shared<Model>(OutputVector{result}, ParameterVector{input_params});
}
std::shared_ptr<Model> orig_function = function->clone();
ov::pass::Manager manager;
manager.register_pass<ov::pass::InitNodeInfo>();
manager.register_pass<ov::intel_gna::pass::SubstituteGNAMaxPool>();
manager.run_passes(function);
ASSERT_NO_THROW(check_rt_info(function));
std::shared_ptr<Model> reference_function;
{
auto input_params = std::make_shared<Parameter>(element::Type_t::f32, Shape{1, 1, 41, 1});
auto transpose_before_const = Constant::create(element::i32, Shape{4}, {0, 2, 3, 1});
auto transpose_before = std::make_shared<Transpose>(input_params, transpose_before_const);
auto max_pool = std::make_shared<ov::intel_gna::op::GNAMaxPool>(transpose_before,
Strides{2, 1},
Shape{0, 0},
Shape{0, 0},
Shape{4, 1});
auto transpose_after_const = Constant::create(element::i32, Shape{4}, {0, 3, 1, 2});
auto transpose_after = std::make_shared<Transpose>(max_pool, transpose_after_const);
const auto result = std::make_shared<Result>(transpose_after);
reference_function = std::make_shared<Model>(OutputVector{result}, ParameterVector{input_params});
}
const FunctionsComparator func_comparator =
FunctionsComparator::with_default().enable(FunctionsComparator::ATTRIBUTES);
const FunctionsComparator::Result result = func_comparator(function, reference_function);
ASSERT_TRUE(result.valid) << result.message;
}

View File

@@ -8,7 +8,7 @@
#include "openvino/opsets/opset12.hpp"
#include "openvino/pass/manager.hpp"
#include "transformations/init_node_info.hpp"
#include "transformations/ts_split.hpp"
#include "transformations/ts_split_backward.hpp"
using namespace ov;
using namespace ov::opset12;

View File

@@ -58,7 +58,7 @@ void ConvolutionLayerTest::SetUp() {
if (targetDevice == CommonTestUtils::DEVICE_GNA) {
auto filter_size = std::accumulate(std::begin(kernel), std::end(kernel), 1, std::multiplies<size_t>());
filter_weights = CommonTestUtils::generate_float_numbers(convOutChannels * inputShape[1] * filter_size,
-0.5f, 0.5f);
-0.1f, 0.1f);
}
auto conv = std::dynamic_pointer_cast<ngraph::opset1::Convolution>(
ngraph::builder::makeConvolution(paramOuts[0], ngPrc, kernel, stride, padBegin,

View File

@@ -56,7 +56,7 @@ void InputConvTest::SetUp() {
std::vector<float> res;
for (std::size_t i = 0; i < out_channels; ++i) {
for (std::size_t j = 0; j < kernel_size; ++j) {
j == 0 ? res.emplace_back(1.0f) : res.emplace_back(0.0f);
j == 0 ? res.emplace_back(0.2f) : res.emplace_back(0.0f);
}
}