Support Multi-PF netdev (Socket Direct)

This series adds support for combining multiple devices (PFs) of the
same port under one netdev instance. Passing traffic through different
devices belonging to different NUMA sockets saves cross-numa traffic and
allows apps running on the same netdev from different numas to still
feel a sense of proximity to the device and achieve improved
performance.

We achieve this by grouping PFs together, and creating the netdev only
once all group members are probed. Symmetrically, we destroy the netdev
once any of the PFs is removed.

The channels are distributed between all devices, a proper configuration
would utilize the correct close numa when working on a certain app/cpu.

We pick one device to be a primary (leader), and it fills a special
role.  The other devices (secondaries) are disconnected from the network
in the chip level (set to silent mode). All RX/TX traffic is steered
through the primary to/from the secondaries.

Currently, we limit the support to PFs only, and up to two devices
(sockets).

V3:
- Fix documentation per Jakubs feedback.
- Fix typos
- Link new documentation in the networking index.rst

V2:
- Add documentation in a new patch.
- Add debugfs in a new patch.
- Add mlx5_ifc bit for MPIR cap check and use it before query.
-----BEGIN PGP SIGNATURE-----

iQEzBAABCAAdFiEEGhZs6bAKwk/OTgTpSD+KveBX+j4FAmXNfcAACgkQSD+KveBX
+j7X2wgA0VToA+d63Gl6meg7o2Txjqt9UkW52rrNmi/1+1F/7X7fnxsQY2EO1oBe
OBJl5/lN2kjbkEdH8j6xRn+5ZaEOaHwuVU+4I2S4Wv68acQrC65IXOC/ZKG9Qm9k
als7XtIByBEKN7w8KDcyx5qV3zU6taSFiCgk24X/miFplo7AsA/XeAKOoIKOH4Zu
kz+zcOyT1yEbpvGDyVOVym/9d8+W1lMbrC+JbnEaenmUJVa5sm7wcWG4BRHMHv4E
gGpURT5OnhisljYvRWogiUCa430AZMGoxOYy3FBESMM1fDcMhesXbURVQtJVRuru
EsnW2OvQ5cQBeQI934pgLdU7dAYESA==
=IGD3
-----END PGP SIGNATURE-----