You can control the behavior of XLIO by configuring:
-
The libxlio.conf file
-
XLIO configuration parameters, which are Linux OS environment variables
-
XLIO extra API
Configuring libxlio.conf
The installation process creates a default configuration file, /etc/libxlio.conf, in which you can define and change the following settings:
-
The target applications or processes to which the configured control settings apply. By default, XLIO control settings are applied to all applications.
-
The transport to be used for the created sockets.
-
The IP addresses and ports in which you want offload.
By default, the configuration file allows XLIO to offload everything except for the DNS server-side protocol (UDP, port 53) which will be handled by the OS.
In the libxlio.conf file:
-
You can define different XLIO control statements for different processes in a single configuration file. Control statements are always applied to the preceding target process statement in the configuration file.
-
Comments start with # and cause the entire line after it to be ignored.
-
Any beginning whitespace is skipped.
-
Any line that is empty is skipped.
-
It is recommended to add comments when making configuration changes.
The following sections describe configuration options in libxlio.conf. For a sample libxlio.conf file, see Example of XLIO Configuration below.
Configuring Target Application or Process
The target process statement specifies the process to which all control statements that appear between this statement and the next target process statement apply.
Each statement specifies a matching rule that all its sub-expressions must evaluate as true (logical and) to apply.
If not provided (default), the statement matches all programs.
The format of the target process statement is:
application-id <program-name|*> <user-defined-id|*>
|
Option |
Description |
|---|---|
|
<program-name|*> |
Define the program name (not including the path) to which the control statements appearing below this statement apply.
|
|
<user-defined-id|*> |
Specify the process ID to which the control statements appearing below this statement apply. You must also set the XLIO_APPLICATION_ID environment variable to the same value as user-defined-id.
|
Configuring Socket Transport Control
Use socket control statements to specify when libxlio will offload AF_INET/SOCK_STREAM or AF_INET/SOCK_DATAGRAM sockets (currently SOCK_RAW is not supported).
Each control statement specifies a matching rule that all its sub-expressions must evaluate as true (logical and) to apply. Statements are evaluated in order of definition according to "first-match".
Socket control statements use the following format:
use <transport> <role> <address|*>:<port range|*>
Where:
|
Option |
Description |
|---|---|
|
transport |
Define the mode of transport:
The default is xlio. |
|
role |
Specify one of the following roles:
|
|
address |
You can specify the local address the server is bind to or the remote server address the client connects to. The syntax for address matching is: <IPv4 address>[/<prefix_length>]|*
|
|
port range |
Define the port range as:
Port range: 0-65536 |
Example of XLIO Configuration
To set the following:
-
Apply the rules to program tcp_lat with ID B1
-
Use XLIO by TCP clients connecting to machines that belong to subnet 192.168.1.*
-
Use OS when TCP server listens to port 5001 of any machine
In libxlio.conf, configure:
application-id tcp-lat B1
use xlio tcp_client 192.168.1.0/24:*:*:*
use os tcp_server *:5001
use os udp_connect *:53
You must also set the XLIO parameter:
XLIO_APPLICATION_ID=B1
XLIO Configuration Parameters
XLIO configuration is performed using environment variables. For the full list of XLIO parameters, please see libxlio README file.
XLIO parameters must be set prior to loading the application with XLIO. You can set the parameters in a system file, which can be run manually or automatically.
All the parameters have defaults that can be modified.
On default startup, the XLIO library prints the XLIO version information, as well as the configuration parameters being used and their values to stderr.
XLIO always logs the values of the following parameters, even when they are equal to the default value:
-
XLIO_TRACELEVEL
-
XLIO_LOG_FILE
For all other parameters, XLIO logs the parameter values only when they are not equal to the default value.
Beta Level Features Configuration Parameters
The following table lists configuration parameters and their possible values for new XLIO Beta level features. The parameters below are disabled by default.
These XLIO features are still experimental and subject to changes. They can help improve performance of multithread applications.
We recommend altering these parameters in a controlled environment until reaching the best performance tuning.
|
XLIO Configuration Parameter |
Description and Examples |
|---|---|
|
XLIO_RING_MIGRATION_RATIO_TX XLIO_RING_MIGRATION_RATIO_RX |
Ring migration ratio is used with the "ring per thread" logic in order to decide when it is beneficial to replace the socket's ring with the ring allocated for the current thread.
|
|
XLIO_RING_LIMIT_PER_INTERFACE |
Limits the number of rings that can be allocated per interface.
XLIO_RX_BUFS might need to be adjusted in order to have enough buffers for all rings in the system. Each ring consumes XLIO_RX_WRE buffers. Use a value of 0 for an unlimited number of rings.
|
|
XLIO_RING_DEV_MEM_TX |
XLIO can use the on-device-memory to store the egress packet if it does not fit into the BF inline buffer. This improves application egress latency by reducing the PCI transactions.
|
|
XLIO_TCP_CC_ALGO |
TCP congestion control algorithm.
|
Loading XLIO Dynamically
XLIO can be loaded using Dynamically Loaded (DL) libraries. These libraries are not automatically loaded at program link time or start-up as with LD_PRELOAD. Instead, there is an API for opening a library, looking up symbols, handling errors, and closing the library.
The example below demonstrates how to load socket() function. Similarly, users should load all other network-related functions as declared in sock-redirect.h:
#include <stdlib.h>
#include <stdio.h>
#include <dlfcn.h>
#include <arpa/inet.h>
#include <sys/socket.h>
typedef int (*socket_fptr_t) (int __domain, int __type, int __protocol);
int main(int argc, const char** argv)
{
void* lib_handle;
socket_fptr_t xlio_socket;
int fd;
lib_handle = dlopen("libxlio.so", RTLD_LAZY);
if (!lib_handle) {
printf("FAILED to load libxlio.so\n");
exit(1);
}
xlio_socket = (socket_fptr_t)dlsym(lib_handle, "socket");
if (xlio_socket == NULL) {
printf("FAILED to load socket()\n");
exit(1);
}
fd = xlio_socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
if (fd < 0) {
printf("FAILED open socket()\n");
exit(1);
}
printf("socket creation succeeded fd = %d\n", fd);
close(fd);
dlclose(lib_handle);
return 0;
}
For more information, please refer to dlopen man page.
For a complete example that includes all the necessary functions, see sockperf’s xlio-redirect.h and xlio_socket-redirect.cpp files.
Last updated: