The Linux kernel dpll subsystem¶
DPLL¶
PLL - Phase Locked Loop is an electronic circuit which syntonizes clock signal of a device with an external clock signal. Effectively enabling device to run on the same clock signal beat as provided on a PLL input.
DPLL - Digital Phase Locked Loop is an integrated circuit which in addition to plain PLL behavior incorporates a digital phase detector and may have digital divider in the loop. As a result, the frequency on DPLL's input and output may be configurable.
Subsystem¶
The main purpose of dpll subsystem is to provide general interface to configure devices that use any kind of Digital PLL and could use different sources of input signal to synchronize to, as well as different types of outputs. The main interface is NETLINK_GENERIC based protocol with an event monitoring multicast group defined.
Device object¶
Single dpll device object means single Digital PLL circuit and bunch of
connected pins.
It reports the supported modes of operation and current status to the
user in response to the do request of netlink command
DPLL_CMD_DEVICE_GET
and list of dplls registered in the subsystem
with dump netlink request of the same command.
Changing the configuration of dpll device is done with do request of
netlink DPLL_CMD_DEVICE_SET
command.
A device handle is DPLL_A_ID
, it shall be provided to get or set
configuration of particular device in the system. It can be obtained
with a DPLL_CMD_DEVICE_GET
dump request or
a DPLL_CMD_DEVICE_ID_GET
do request, where the one must provide
attributes that result in single device match.
Pin object¶
A pin is amorphic object which represents either input or output, it
could be internal component of the device, as well as externally
connected.
The number of pins per dpll vary, but usually multiple pins shall be
provided for a single dpll device.
Pin's properties, capabilities and status is provided to the user in
response to do request of netlink DPLL_CMD_PIN_GET
command.
It is also possible to list all the pins that were registered in the
system with dump request of DPLL_CMD_PIN_GET
command.
Configuration of a pin can be changed by do request of netlink
DPLL_CMD_PIN_SET
command.
Pin handle is a DPLL_A_PIN_ID
, it shall be provided to get or set
configuration of particular pin in the system. It can be obtained with
DPLL_CMD_PIN_GET
dump request or DPLL_CMD_PIN_ID_GET
do
request, where user provides attributes that result in single pin match.
Pin selection¶
In general, selected pin (the one which signal is driving the dpll
device) can be obtained from DPLL_A_PIN_STATE
attribute, and only
one pin shall be in DPLL_PIN_STATE_CONNECTED
state for any dpll
device.
Pin selection can be done either manually or automatically, depending
on hardware capabilities and active dpll device work mode
(DPLL_A_MODE
attribute). The consequence is that there are
differences for each mode in terms of available pin states, as well as
for the states the user can request for a dpll device.
In manual mode (DPLL_MODE_MANUAL
) the user can request or receive
one of following pin states:
DPLL_PIN_STATE_CONNECTED
- the pin is used to drive dpll deviceDPLL_PIN_STATE_DISCONNECTED
- the pin is not used to drive dpll device
In automatic mode (DPLL_MODE_AUTOMATIC
) the user can request or
receive one of following pin states:
DPLL_PIN_STATE_SELECTABLE
- the pin shall be considered as valid input for automatic selection algorithmDPLL_PIN_STATE_DISCONNECTED
- the pin shall be not considered as a valid input for automatic selection algorithm
In automatic mode (DPLL_MODE_AUTOMATIC
) the user can only receive
pin state DPLL_PIN_STATE_CONNECTED
once automatic selection
algorithm locks a dpll device with one of the inputs.
MUX-type pins¶
A pin can be MUX-type, it aggregates child pins and serves as a pin
multiplexer. One or more pins are registered with MUX-type instead of
being directly registered to a dpll device.
Pins registered with a MUX-type pin provide user with additional nested
attribute DPLL_A_PIN_PARENT_PIN
for each parent they were registered
with.
If a pin was registered with multiple parent pins, they behave like a
multiple output multiplexer. In this case output of a
DPLL_CMD_PIN_GET
would contain multiple pin-parent nested
attributes with current state related to each parent, like:
'pin': [{{
'clock-id': 282574471561216,
'module-name': 'ice',
'capabilities': 4,
'id': 13,
'parent-pin': [
{'parent-id': 2, 'state': 'connected'},
{'parent-id': 3, 'state': 'disconnected'}
],
'type': 'synce-eth-port'
}}]
Only one child pin can provide its signal to the parent MUX-type pin at
a time, the selection is done by requesting change of a child pin state
on desired parent, with the use of DPLL_A_PIN_PARENT
nested
attribute. Example of netlink set state on parent pin message format:
DPLL_A_PIN_ID
child pin id
DPLL_A_PIN_PARENT_PIN
nested attribute for requesting configuration related to parent pin
DPLL_A_PIN_PARENT_ID
parent pin id
DPLL_A_PIN_STATE
requested pin state on parent
Pin priority¶
Some devices might offer a capability of automatic pin selection mode
(enum value DPLL_MODE_AUTOMATIC
of DPLL_A_MODE
attribute).
Usually, automatic selection is performed on the hardware level, which
means only pins directly connected to the dpll can be used for automatic
input pin selection.
In automatic selection mode, the user cannot manually select a input
pin for the device, instead the user shall provide all directly
connected pins with a priority DPLL_A_PIN_PRIO
, the device would
pick a highest priority valid signal and use it to control the DPLL
device. Example of netlink set priority on parent pin message format:
DPLL_A_PIN_ID
configured pin id
DPLL_A_PIN_PARENT_DEVICE
nested attribute for requesting configuration related to parent dpll device
DPLL_A_PIN_PARENT_ID
parent dpll device id
DPLL_A_PIN_PRIO
requested pin prio on parent dpll
Child pin of MUX-type pin is not capable of automatic input pin selection,
in order to configure active input of a MUX-type pin, the user needs to
request desired pin state of the child pin on the parent pin,
as described in the MUX-type pins
chapter.
Phase offset measurement and adjustment¶
Device may provide ability to measure a phase difference between signals
on a pin and its parent dpll device. If pin-dpll phase offset measurement
is supported, it shall be provided with DPLL_A_PIN_PHASE_OFFSET
attribute for each parent dpll device.
Device may also provide ability to adjust a signal phase on a pin.
If pin phase adjustment is supported, minimal and maximal values that pin
handle shall be provide to the user on DPLL_CMD_PIN_GET
respond
with DPLL_A_PIN_PHASE_ADJUST_MIN
and DPLL_A_PIN_PHASE_ADJUST_MAX
attributes. Configured phase adjust value is provided with
DPLL_A_PIN_PHASE_ADJUST
attribute of a pin, and value change can be
requested with the same attribute with DPLL_CMD_PIN_SET
command.
DPLL_A_PIN_ID
configured pin id
DPLL_A_PIN_PHASE_ADJUST_MIN
attr minimum value of phase adjustment
DPLL_A_PIN_PHASE_ADJUST_MAX
attr maximum value of phase adjustment
DPLL_A_PIN_PHASE_ADJUST
attr configured value of phase adjustment on parent dpll device
DPLL_A_PIN_PARENT_DEVICE
nested attribute for requesting configuration on given parent dpll device
DPLL_A_PIN_PARENT_ID
parent dpll device id
DPLL_A_PIN_PHASE_OFFSET
attr measured phase difference between a pin and parent dpll device
All phase related values are provided in pico seconds, which represents time difference between signals phase. The negative value means that phase of signal on pin is earlier in time than dpll's signal. Positive value means that phase of signal on pin is later in time than signal of a dpll.
Phase adjust (also min and max) values are integers, but measured phase
offset values are fractional with 3-digit decimal places and shell be
divided with DPLL_PIN_PHASE_OFFSET_DIVIDER
to get integer part and
modulo divided to get fractional part.
Configuration commands group¶
Configuration commands are used to get information about registered dpll devices (and pins), as well as set configuration of device or pins. As dpll devices must be abstracted and reflect real hardware, there is no way to add new dpll device via netlink from user space and each device should be registered by its driver.
All netlink commands require GENL_ADMIN_PERM
. This is to prevent
any spamming/DoS from unauthorized userspace applications.
List of netlink commands with possible attributes¶
Constants identifying command types for dpll device uses a
DPLL_CMD_
prefix and suffix according to command purpose.
The dpll device related attributes use a DPLL_A_
prefix and
suffix according to attribute purpose.
DPLL_CMD_DEVICE_ID_GET
command to get device ID
DPLL_A_MODULE_NAME
attr module name of registerer
DPLL_A_CLOCK_ID
attr Unique Clock Identifier (EUI-64), as defined by the IEEE 1588 standard
DPLL_A_TYPE
attr type of dpll device
DPLL_CMD_DEVICE_GET
command to get device info or dump list of available devices
DPLL_A_ID
attr unique dpll device ID
DPLL_A_MODULE_NAME
attr module name of registerer
DPLL_A_CLOCK_ID
attr Unique Clock Identifier (EUI-64), as defined by the IEEE 1588 standard
DPLL_A_MODE
attr selection mode
DPLL_A_MODE_SUPPORTED
attr available selection modes
DPLL_A_LOCK_STATUS
attr dpll device lock status
DPLL_A_TEMP
attr device temperature info
DPLL_A_TYPE
attr type of dpll device
DPLL_CMD_DEVICE_SET
command to set dpll device config
DPLL_A_ID
attr internal dpll device index
DPLL_A_MODE
attr selection mode to configure
Constants identifying command types for pins uses a
DPLL_CMD_PIN_
prefix and suffix according to command purpose.
The pin related attributes use a DPLL_A_PIN_
prefix and suffix
according to attribute purpose.
DPLL_CMD_PIN_ID_GET
command to get pin ID
DPLL_A_PIN_MODULE_NAME
attr module name of registerer
DPLL_A_PIN_CLOCK_ID
attr Unique Clock Identifier (EUI-64), as defined by the IEEE 1588 standard
DPLL_A_PIN_BOARD_LABEL
attr pin board label provided by registerer
DPLL_A_PIN_PANEL_LABEL
attr pin panel label provided by registerer
DPLL_A_PIN_PACKAGE_LABEL
attr pin package label provided by registerer
DPLL_A_PIN_TYPE
attr type of a pin
DPLL_CMD_PIN_GET
command to get pin info or dump list of available pins
DPLL_A_PIN_ID
attr unique a pin ID
DPLL_A_PIN_MODULE_NAME
attr module name of registerer
DPLL_A_PIN_CLOCK_ID
attr Unique Clock Identifier (EUI-64), as defined by the IEEE 1588 standard
DPLL_A_PIN_BOARD_LABEL
attr pin board label provided by registerer
DPLL_A_PIN_PANEL_LABEL
attr pin panel label provided by registerer
DPLL_A_PIN_PACKAGE_LABEL
attr pin package label provided by registerer
DPLL_A_PIN_TYPE
attr type of a pin
DPLL_A_PIN_FREQUENCY
attr current frequency of a pin
DPLL_A_PIN_FREQUENCY_SUPPORTED
nested attr provides supported frequencies
DPLL_A_PIN_ANY_FREQUENCY_MIN
attr minimum value of frequency
DPLL_A_PIN_ANY_FREQUENCY_MAX
attr maximum value of frequency
DPLL_A_PIN_PHASE_ADJUST_MIN
attr minimum value of phase adjustment
DPLL_A_PIN_PHASE_ADJUST_MAX
attr maximum value of phase adjustment
DPLL_A_PIN_PHASE_ADJUST
attr configured value of phase adjustment on parent device
DPLL_A_PIN_PARENT_DEVICE
nested attr for each parent device the pin is connected with
DPLL_A_PIN_PARENT_ID
attr parent dpll device id
DPLL_A_PIN_PRIO
attr priority of pin on the dpll device
DPLL_A_PIN_STATE
attr state of pin on the parent dpll device
DPLL_A_PIN_DIRECTION
attr direction of a pin on the parent dpll device
DPLL_A_PIN_PHASE_OFFSET
attr measured phase difference between a pin and parent dpll
DPLL_A_PIN_PARENT_PIN
nested attr for each parent pin the pin is connected with
DPLL_A_PIN_PARENT_ID
attr parent pin id
DPLL_A_PIN_STATE
attr state of pin on the parent pin
DPLL_A_PIN_CAPABILITIES
attr bitmask of pin capabilities
DPLL_CMD_PIN_SET
command to set pins configuration
DPLL_A_PIN_ID
attr unique a pin ID
DPLL_A_PIN_FREQUENCY
attr requested frequency of a pin
DPLL_A_PIN_PHASE_ADJUST
attr requested value of phase adjustment on parent device
DPLL_A_PIN_PARENT_DEVICE
nested attr for each parent dpll device configuration request
DPLL_A_PIN_PARENT_ID
attr parent dpll device id
DPLL_A_PIN_DIRECTION
attr requested direction of a pin
DPLL_A_PIN_PRIO
attr requested priority of pin on the dpll device
DPLL_A_PIN_STATE
attr requested state of pin on the dpll device
DPLL_A_PIN_PARENT_PIN
nested attr for each parent pin configuration request
DPLL_A_PIN_PARENT_ID
attr parent pin id
DPLL_A_PIN_STATE
attr requested state of pin on parent pin
Netlink dump requests¶
The DPLL_CMD_DEVICE_GET
and DPLL_CMD_PIN_GET
commands are
capable of dump type netlink requests, in which case the response is in
the same format as for their do
request, but every device or pin
registered in the system is returned.
SET commands format¶
DPLL_CMD_DEVICE_SET
- to target a dpll device, the user provides
DPLL_A_ID
, which is unique identifier of dpll device in the system,
as well as parameter being configured (DPLL_A_MODE
).
DPLL_CMD_PIN_SET
- to target a pin user must provide a
DPLL_A_PIN_ID
, which is unique identifier of a pin in the system.
Also configured pin parameters must be added.
If DPLL_A_PIN_FREQUENCY
is configured, this affects all the dpll
devices that are connected with the pin, that is why frequency attribute
shall not be enclosed in DPLL_A_PIN_PARENT_DEVICE
.
Other attributes: DPLL_A_PIN_PRIO
, DPLL_A_PIN_STATE
or
DPLL_A_PIN_DIRECTION
must be enclosed in
DPLL_A_PIN_PARENT_DEVICE
as their configuration relates to only one
of parent dplls, targeted by DPLL_A_PIN_PARENT_ID
attribute which is
also required inside that nest.
For MUX-type pins the DPLL_A_PIN_STATE
attribute is configured in
similar way, by enclosing required state in DPLL_A_PIN_PARENT_PIN
nested attribute and targeted parent pin id in DPLL_A_PIN_PARENT_ID
.
In general, it is possible to configure multiple parameters at once, but internally each parameter change will be invoked separately, where order of configuration is not guaranteed by any means.
Configuration pre-defined enums¶
-
enum dpll_mode¶
working modes a dpll can support, differentiates if and how dpll selects one of its inputs to syntonize with it, valid values for DPLL_A_MODE attribute
Constants
DPLL_MODE_MANUAL
input can be only selected by sending a request to dpll
DPLL_MODE_AUTOMATIC
highest prio input pin auto selected by dpll
-
enum dpll_lock_status¶
provides information of dpll device lock status, valid values for DPLL_A_LOCK_STATUS attribute
Constants
DPLL_LOCK_STATUS_UNLOCKED
dpll was not yet locked to any valid input (or forced by setting DPLL_A_MODE to DPLL_MODE_DETACHED)
DPLL_LOCK_STATUS_LOCKED
dpll is locked to a valid signal, but no holdover available
DPLL_LOCK_STATUS_LOCKED_HO_ACQ
dpll is locked and holdover acquired
DPLL_LOCK_STATUS_HOLDOVER
dpll is in holdover state - lost a valid lock or was forced by disconnecting all the pins (latter possible only when dpll lock-state was already DPLL_LOCK_STATUS_LOCKED_HO_ACQ, if dpll lock-state was not DPLL_LOCK_STATUS_LOCKED_HO_ACQ, the dpll's lock-state shall remain DPLL_LOCK_STATUS_UNLOCKED)
-
enum dpll_type¶
type of dpll, valid values for DPLL_A_TYPE attribute
Constants
DPLL_TYPE_PPS
dpll produces Pulse-Per-Second signal
DPLL_TYPE_EEC
dpll drives the Ethernet Equipment Clock
-
enum dpll_pin_type¶
defines possible types of a pin, valid values for DPLL_A_PIN_TYPE attribute
Constants
DPLL_PIN_TYPE_MUX
aggregates another layer of selectable pins
DPLL_PIN_TYPE_EXT
external input
DPLL_PIN_TYPE_SYNCE_ETH_PORT
ethernet port PHY's recovered clock
DPLL_PIN_TYPE_INT_OSCILLATOR
device internal oscillator
DPLL_PIN_TYPE_GNSS
GNSS recovered clock
-
enum dpll_pin_direction¶
defines possible direction of a pin, valid values for DPLL_A_PIN_DIRECTION attribute
Constants
DPLL_PIN_DIRECTION_INPUT
pin used as a input of a signal
DPLL_PIN_DIRECTION_OUTPUT
pin used to output the signal
-
enum dpll_pin_state¶
defines possible states of a pin, valid values for DPLL_A_PIN_STATE attribute
Constants
DPLL_PIN_STATE_CONNECTED
pin connected, active input of phase locked loop
DPLL_PIN_STATE_DISCONNECTED
pin disconnected, not considered as a valid input
DPLL_PIN_STATE_SELECTABLE
pin enabled for automatic input selection
-
enum dpll_pin_capabilities¶
defines possible capabilities of a pin, valid flags on DPLL_A_PIN_CAPABILITIES attribute
Constants
DPLL_PIN_CAPABILITIES_DIRECTION_CAN_CHANGE
pin direction can be changed
DPLL_PIN_CAPABILITIES_PRIORITY_CAN_CHANGE
pin priority can be changed
DPLL_PIN_CAPABILITIES_STATE_CAN_CHANGE
pin state can be changed
Notifications¶
dpll device can provide notifications regarding status changes of the
device, i.e. lock status changes, input/output changes or other alarms.
There is one multicast group that is used to notify user-space apps via
netlink socket: DPLL_MCGRP_MONITOR
Notifications messages:
DPLL_CMD_DEVICE_CREATE_NTF
dpll device was created
DPLL_CMD_DEVICE_DELETE_NTF
dpll device was deleted
DPLL_CMD_DEVICE_CHANGE_NTF
dpll device has changed
DPLL_CMD_PIN_CREATE_NTF
dpll pin was created
DPLL_CMD_PIN_DELETE_NTF
dpll pin was deleted
DPLL_CMD_PIN_CHANGE_NTF
dpll pin has changed
Events format is the same as for the corresponding get command.
Format of DPLL_CMD_DEVICE_
events is the same as response of
DPLL_CMD_DEVICE_GET
.
Format of DPLL_CMD_PIN_
events is same as response of
DPLL_CMD_PIN_GET
.
Device driver implementation¶
Device is allocated by dpll_device_get() call. Second call with the same arguments will not create new object but provides pointer to previously created device for given arguments, it also increases refcount of that object. Device is deallocated by dpll_device_put() call, which first decreases the refcount, once refcount is cleared the object is destroyed.
Device should implement set of operations and register device via dpll_device_register() at which point it becomes available to the users. Multiple driver instances can obtain reference to it with dpll_device_get(), as well as register dpll device with their own ops and priv.
The pins are allocated separately with dpll_pin_get(), it works similarly to dpll_device_get(). Function first creates object and then for each call with the same arguments only the object refcount increases. Also dpll_pin_put() works similarly to dpll_device_put().
A pin can be registered with parent dpll device or parent pin, depending on hardware needs. Each registration requires registerer to provide set of pin callbacks, and private data pointer for calling them:
dpll_pin_register() - register pin with a dpll device,
dpll_pin_on_pin_register() - register pin with another MUX type pin.
Notifications of adding or removing dpll devices are created within subsystem itself. Notifications about registering/deregistering pins are also invoked by the subsystem. Notifications about status changes either of dpll device or a pin are invoked in two ways:
after successful change was requested on dpll subsystem, the subsystem calls corresponding notification,
requested by device driver with dpll_device_change_ntf() or dpll_pin_change_ntf() when driver informs about the status change.
The device driver using dpll interface is not required to implement all the callback operation. Nevertheless, there are few required to be implemented. Required dpll device level callback operations:
.mode_get
,.lock_status_get
.
Required pin level callback operations:
.state_on_dpll_get
(pins registered with dpll device),.state_on_pin_get
(pins registered with parent pin),.direction_get
.
Every other operation handler is checked for existence and
-EOPNOTSUPP
is returned in case of absence of specific handler.
The simplest implementation is in the OCP TimeCard driver. The ops structures are defined like this:
static const struct dpll_device_ops dpll_ops = {
.lock_status_get = ptp_ocp_dpll_lock_status_get,
.mode_get = ptp_ocp_dpll_mode_get,
.mode_supported = ptp_ocp_dpll_mode_supported,
};
static const struct dpll_pin_ops dpll_pins_ops = {
.frequency_get = ptp_ocp_dpll_frequency_get,
.frequency_set = ptp_ocp_dpll_frequency_set,
.direction_get = ptp_ocp_dpll_direction_get,
.direction_set = ptp_ocp_dpll_direction_set,
.state_on_dpll_get = ptp_ocp_dpll_state_get,
};
The registration part is then looks like this part:
clkid = pci_get_dsn(pdev);
bp->dpll = dpll_device_get(clkid, 0, THIS_MODULE);
if (IS_ERR(bp->dpll)) {
err = PTR_ERR(bp->dpll);
dev_err(&pdev->dev, "dpll_device_alloc failed\n");
goto out;
}
err = dpll_device_register(bp->dpll, DPLL_TYPE_PPS, &dpll_ops, bp);
if (err)
goto out;
for (i = 0; i < OCP_SMA_NUM; i++) {
bp->sma[i].dpll_pin = dpll_pin_get(clkid, i, THIS_MODULE, &bp->sma[i].dpll_prop);
if (IS_ERR(bp->sma[i].dpll_pin)) {
err = PTR_ERR(bp->dpll);
goto out_dpll;
}
err = dpll_pin_register(bp->dpll, bp->sma[i].dpll_pin, &dpll_pins_ops,
&bp->sma[i]);
if (err) {
dpll_pin_put(bp->sma[i].dpll_pin);
goto out_dpll;
}
}
In the error path we have to rewind every allocation in the reverse order:
while (i) {
--i;
dpll_pin_unregister(bp->dpll, bp->sma[i].dpll_pin, &dpll_pins_ops, &bp->sma[i]);
dpll_pin_put(bp->sma[i].dpll_pin);
}
dpll_device_put(bp->dpll);
More complex example can be found in Intel's ICE driver or nVidia's mlx5 driver.
SyncE enablement¶
For SyncE enablement it is required to allow control over dpll device
for a software application which monitors and configures the inputs of
dpll device in response to current state of a dpll device and its
inputs.
In such scenario, dpll device input signal shall be also configurable
to drive dpll with signal recovered from the PHY netdevice.
This is done by exposing a pin to the netdevice - attaching pin to the
netdevice itself with
dpll_netdev_pin_set(struct net_device *dev, struct dpll_pin *dpll_pin)
.
Exposed pin id handle DPLL_A_PIN_ID
is then identifiable by the user
as it is attached to rtnetlink respond to get RTM_NEWLINK
command in
nested attribute IFLA_DPLL_PIN
.