When performing a backup of a Linux client to a remote backup server, the backup fails with the error message “Timed out waiting for input”.

This only appears to fail when writing data from the /var filesystem. What is causing this?

ANSWER

The “Timed out waiting for input” error message is produced by the SBAdmin remote communications utility (strexec) that is responsible for sending backup data to the remote server. The message is displayed, and the backup fails, when strexec has not received data to send to the server for a timeout of 10 minutes. The strexec utility gets data to send to the remote server from the backup process running on the client.

This error message is most commonly found when performing a backup of a Linux x86_64 client, and the file /var/log/lastlog is found to be very large (some have reported it as 1.2TB) on the client.

HANDLING /var/log/lastlog

The file /var/log/lastlog is a database which contains information on the last login of each user. It is a sparse file. Therefore, it’s size when shown with “ls -l” may display it as very large, but it’s size on disk is typically much smaller. The real size can be displayed with “ls -s” or “du -s”. Most often it will appear very large if there are system users configured with very high UID’s.

In part, SBAdmin utilizes the backup client’s “tar” command when packaging files to include in a backup. When the “tar” command packages large sparse files like /var/log/lastlog, it will read through the entire file before outputting the data to stdout. This behavior is even found when using the “–sparse” or “-S” flag with “tar”, which is documented to handle sparse files efficiently.

The problem this causes with SBAdmin is most often found when performing a backup of a client and storing the backup on a remote server. The SBAdmin communication program (strexec) is waiting for input from the “tar” command on stdout, and will continue to wait for a timeout of 10 minutes. When no data is passed from “tar” to “strexec” for 10 minutes the backup will fail and display the error message “Timed out waiting on input”.

If your backup is experiencing this problem, there are 3 methods to overcome the issue: