| .. SPDX-License-Identifier: GPL-2.0 | |
| ==================== | |
| Interface statistics | |
| ==================== | |
| Overview | |
| ======== | |
| This document is a guide to Linux network interface statistics. | |
| There are three main sources of interface statistics in Linux: | |
| - standard interface statistics based on | |
| :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`; | |
| - protocol-specific statistics; and | |
| - driver-defined statistics available via ethtool. | |
| Standard interface statistics | |
| ----------------------------- | |
| There are multiple interfaces to reach the standard statistics. | |
| Most commonly used is the `ip` command from `iproute2`:: | |
| $ ip -s -s link show dev ens4u1u1 | |
| 6: ens4u1u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000 | |
| link/ether 48:2a:e3:4c:b1:d1 brd ff:ff:ff:ff:ff:ff | |
| RX: bytes packets errors dropped overrun mcast | |
| 74327665117 69016965 0 0 0 0 | |
| RX errors: length crc frame fifo missed | |
| 0 0 0 0 0 | |
| TX: bytes packets errors dropped carrier collsns | |
| 21405556176 44608960 0 0 0 0 | |
| TX errors: aborted fifo window heartbeat transns | |
| 0 0 0 0 128 | |
| altname enp58s0u1u1 | |
| Note that `-s` has been specified twice to see all members of | |
| :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`. | |
| If `-s` is specified once the detailed errors won't be shown. | |
| `ip` supports JSON formatting via the `-j` option. | |
| Queue statistics | |
| ~~~~~~~~~~~~~~~~ | |
| Queue statistics are accessible via the netdev netlink family. | |
| Currently no widely distributed CLI exists to access those statistics. | |
| Kernel development tools (ynl) can be used to experiment with them, | |
| see `Documentation/userspace-api/netlink/intro-specs.rst`. | |
| Protocol-specific statistics | |
| ---------------------------- | |
| Protocol-specific statistics are exposed via relevant interfaces, | |
| the same interfaces as are used to configure them. | |
| ethtool | |
| ~~~~~~~ | |
| Ethtool exposes common low-level statistics. | |
| All the standard statistics are expected to be maintained | |
| by the device, not the driver (as opposed to driver-defined stats | |
| described in the next section which mix software and hardware stats). | |
| For devices which contain unmanaged | |
| switches (e.g. legacy SR-IOV or multi-host NICs) the events counted | |
| may not pertain exclusively to the packets destined to | |
| the local host interface. In other words the events may | |
| be counted at the network port (MAC/PHY blocks) without separation | |
| for different host side (PCIe) devices. Such ambiguity must not | |
| be present when internal switch is managed by Linux (so called | |
| switchdev mode for NICs). | |
| Standard ethtool statistics can be accessed via the interfaces used | |
| for configuration. For example ethtool interface used | |
| to configure pause frames can report corresponding hardware counters:: | |
| $ ethtool --include-statistics -a eth0 | |
| Pause parameters for eth0: | |
| Autonegotiate: on | |
| RX: on | |
| TX: on | |
| Statistics: | |
| tx_pause_frames: 1 | |
| rx_pause_frames: 1 | |
| General Ethernet statistics not associated with any particular | |
| functionality are exposed via ``ethtool -S $ifc`` by specifying | |
| the ``--groups`` parameter:: | |
| $ ethtool -S eth0 --groups eth-phy eth-mac eth-ctrl rmon | |
| Stats for eth0: | |
| eth-phy-SymbolErrorDuringCarrier: 0 | |
| eth-mac-FramesTransmittedOK: 1 | |
| eth-mac-FrameTooLongErrors: 1 | |
| eth-ctrl-MACControlFramesTransmitted: 1 | |
| eth-ctrl-MACControlFramesReceived: 0 | |
| eth-ctrl-UnsupportedOpcodesReceived: 1 | |
| rmon-etherStatsUndersizePkts: 1 | |
| rmon-etherStatsJabbers: 0 | |
| rmon-rx-etherStatsPkts64Octets: 1 | |
| rmon-rx-etherStatsPkts65to127Octets: 0 | |
| rmon-rx-etherStatsPkts128to255Octets: 0 | |
| rmon-tx-etherStatsPkts64Octets: 2 | |
| rmon-tx-etherStatsPkts65to127Octets: 3 | |
| rmon-tx-etherStatsPkts128to255Octets: 0 | |
| Driver-defined statistics | |
| ------------------------- | |
| Driver-defined ethtool statistics can be dumped using `ethtool -S $ifc`, e.g.:: | |
| $ ethtool -S ens4u1u1 | |
| NIC statistics: | |
| tx_single_collisions: 0 | |
| tx_multi_collisions: 0 | |
| uAPIs | |
| ===== | |
| procfs | |
| ------ | |
| The historical `/proc/net/dev` text interface gives access to the list | |
| of interfaces as well as their statistics. | |
| Note that even though this interface is using | |
| :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` | |
| internally it combines some of the fields. | |
| sysfs | |
| ----- | |
| Each device directory in sysfs contains a `statistics` directory (e.g. | |
| `/sys/class/net/lo/statistics/`) with files corresponding to | |
| members of :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`. | |
| This simple interface is convenient especially in constrained/embedded | |
| environments without access to tools. However, it's inefficient when | |
| reading multiple stats as it internally performs a full dump of | |
| :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` | |
| and reports only the stat corresponding to the accessed file. | |
| Sysfs files are documented in | |
| Documentation/ABI/testing/sysfs-class-net-statistics. | |
| netlink | |
| ------- | |
| `rtnetlink` (`NETLINK_ROUTE`) is the preferred method of accessing | |
| :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` stats. | |
| Statistics are reported both in the responses to link information | |
| requests (`RTM_GETLINK`) and statistic requests (`RTM_GETSTATS`, | |
| when `IFLA_STATS_LINK_64` bit is set in the `.filter_mask` of the request). | |
| netdev (netlink) | |
| ~~~~~~~~~~~~~~~~ | |
| `netdev` generic netlink family allows accessing page pool and per queue | |
| statistics. | |
| ethtool | |
| ------- | |
| Ethtool IOCTL interface allows drivers to report implementation | |
| specific statistics. Historically it has also been used to report | |
| statistics for which other APIs did not exist, like per-device-queue | |
| statistics, or standard-based statistics (e.g. RFC 2863). | |
| Statistics and their string identifiers are retrieved separately. | |
| Identifiers via `ETHTOOL_GSTRINGS` with `string_set` set to `ETH_SS_STATS`, | |
| and values via `ETHTOOL_GSTATS`. User space should use `ETHTOOL_GDRVINFO` | |
| to retrieve the number of statistics (`.n_stats`). | |
| ethtool-netlink | |
| --------------- | |
| Ethtool netlink is a replacement for the older IOCTL interface. | |
| Protocol-related statistics can be requested in get commands by setting | |
| the `ETHTOOL_FLAG_STATS` flag in `ETHTOOL_A_HEADER_FLAGS`. Currently | |
| statistics are supported in the following commands: | |
| - `ETHTOOL_MSG_FEC_GET` | |
| - `ETHTOOL_MSG_LINKSTATE_GET` | |
| - `ETHTOOL_MSG_MM_GET` | |
| - `ETHTOOL_MSG_PAUSE_GET` | |
| - `ETHTOOL_MSG_TSINFO_GET` | |
| debugfs | |
| ------- | |
| Some drivers expose extra statistics via `debugfs`. | |
| struct rtnl_link_stats64 | |
| ======================== | |
| .. kernel-doc:: include/uapi/linux/if_link.h | |
| :identifiers: rtnl_link_stats64 | |
| Notes for driver authors | |
| ======================== | |
| Drivers should report all statistics which have a matching member in | |
| :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` exclusively | |
| via `.ndo_get_stats64`. Reporting such standard stats via ethtool | |
| or debugfs will not be accepted. | |
| Drivers must ensure best possible compliance with | |
| :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`. | |
| Please note for example that detailed error statistics must be | |
| added into the general `rx_error` / `tx_error` counters. | |
| The `.ndo_get_stats64` callback can not sleep because of accesses | |
| via `/proc/net/dev`. If driver may sleep when retrieving the statistics | |
| from the device it should do so periodically asynchronously and only return | |
| a recent copy from `.ndo_get_stats64`. Ethtool interrupt coalescing interface | |
| allows setting the frequency of refreshing statistics, if needed. | |
| Retrieving ethtool statistics is a multi-syscall process, drivers are advised | |
| to keep the number of statistics constant to avoid race conditions with | |
| user space trying to read them. | |
| Statistics must persist across routine operations like bringing the interface | |
| down and up. | |
| Kernel-internal data structures | |
| ------------------------------- | |
| The following structures are internal to the kernel, their members are | |
| translated to netlink attributes when dumped. Fields are pre-initialized | |
| to ``ETHTOOL_STAT_NOT_SET`` (by ``ethtool_stats_init()``); drivers must | |
| leave fields they do not report at that value rather than overwriting | |
| them with 0. | |
| - ``struct ethtool_eth_ctrl_stats`` | |
| - ``struct ethtool_eth_mac_stats`` | |
| - ``struct ethtool_eth_phy_stats`` | |
| - ``struct ethtool_fec_hist`` | |
| - ``struct ethtool_fec_stats`` | |
| - ``struct ethtool_link_ext_stats`` | |
| - ``struct ethtool_mm_stats`` | |
| - ``struct ethtool_pause_stats`` | |
| - ``struct ethtool_phy_stats`` | |
| - ``struct ethtool_rmon_stats`` | |
| - ``struct ethtool_ts_stats`` |