mlx5-socket-direct V2

V2: Added documentation and debugfs

Tariq Says:
===========
Support Socket-Direct multi-dev netdev

This series adds support for combining multiple devices (PFs) of the
same port under one netdev instance. Passing traffic through different
devices belonging to different NUMA sockets saves cross-numa traffic and
allows apps running on the same netdev from different numas to still
feel a sense of proximity to the device and achieve improved
performance.

We achieve this by grouping PFs together, and creating the netdev only
once all group members are probed. Symmetrically, we destroy the netdev
once any of the PFs is removed.

The channels are distributed between all devices, a proper configuration
would utilize the correct close numa when working on a certain app/cpu.

We pick one device to be a primary (leader), and it fills a special
role.  The other devices (secondaries) are disconnected from the network
in the chip level (set to silent mode). All RX/TX traffic is steered
through the primary to/from the secondaries.

Currently, we limit the support to PFs only, and up to two devices
(sockets).

===========
-----BEGIN PGP SIGNATURE-----

iQEzBAABCAAdFiEEGhZs6bAKwk/OTgTpSD+KveBX+j4FAmXETgAACgkQSD+KveBX
+j7/vgf9HIiy/kgLnK3z+mdCcrPW8eq0NB2kVTbbsHffX/LRfFP2AnqirBN6SNsw
Jchcu9Dn5lAKabs3wF5u0LXUgDRr718lgSoH9GGsC13L8DwpB0hVsf8erxAfQaHU
N118czI7ZkZZLFBJgLywGYZwh5k/MFrUXmZfzwOKtihvEVxvD9EH9INgWO03h5zI
hUufl+U0LjMHdOuITYUmUlQwiOaXTk4aCvFzp/uUJHgQ8oJmeg0DFb5qw8IPaRib
Je7MhtMTjJjnACyjVPuqRk4o/hIfLOI7nQGaA5Z1VqpgWaN455YmYBf/IxFA1aMp
7BK7mKF3lqjRIUcdPQL+7AKDZvcfoQ==
=wSYb
-----END PGP SIGNATURE-----