Whitelist sender in Zimbra

Posted: November 6th, 2015 | Author: | Filed under: Mail Server, Postfix, Zimbra | No Comments »

Email messages from of our collaborator were rejected due to reject_invalid_helo test in Postfix. Ideally, configuration of their sending server should be corrected. This, however, wasn’t happening in this case, or it wasn’t happening fast enough. Thus, the solution, or rather workaround, was to whitelist the server.
Read the rest of this entry »


Disk quota check during user login

Posted: April 9th, 2015 | Author: | Filed under: Linux, Storage | No Comments »

Our users are often unaware of their home directory disk usage and get surprised where they are over their assigned disk quota and cannot write anymore. The script, that follows, is placed in the system /etc/profile.d directory with the name homequota.sh. It checks disk usage and displays information about disk space available under user’s quota. Read the rest of this entry »


Distribution of undetermined indices in Illumina Hiseq experiments

Posted: March 23rd, 2015 | Author: | Filed under: Sequencing | No Comments »

It is often that one wants to look at distribution of undetermined indices in Illumina Hiseq experiments to spot problems with experiment’s sample sheet or with the library itself. These indices are stored in fastq file(s) in the undetermined_indices folder. Below is the Bash script that processes these fastq files (fastq files can be in gzip’ed and/or uncompressed forms), and prints distribution of indices to stdout in a form of comma-separated values. If the script is named “index_stats”, one calls it from the undetermined_indices folder as:

   index_stats > lane2_undetermined.csv

Latest versions of GNU sort program support parallel sorting. The script as presented below, runs GNU sort with 8 threads. It also utilizes parallel unpigz with 8 cores/processors to uncompress fastq.gz files, if unpigz is present. Otherwise, it uses single core gunzip command.
Read the rest of this entry »


Back up of large filesystem with a very large number of files

Posted: June 6th, 2014 | Author: | Filed under: Backup, Linux, Storage | Comments Off on Back up of large filesystem with a very large number of files

Backing up large filesystems with very large numbers of files and directories is challenging. I routinely deal with 20-40TB filesystems with millions, sometimes tens of millions, of files. Filesystem with Unix home directories in a place with large number of users is an example of such a filesystem. The problem is more of backup speed than capacity.

Some commercial backup packages can do a better job here, but if you rely on free backup solutions, the situation is a bit more challenging. I have tried quite a few open-source backup programs, and performance of most of them is not acceptable in this type of environment. However, I have identified a few approaches which performance comes as acceptable, barely acceptable but acceptable from my perspective. Read the rest of this entry »


Monitor disks behind Dell RAID controllers without OpenManage

Posted: August 9th, 2013 | Author: | Filed under: Linux, Storage | No Comments »

When one runs non-RHEL linux on modern Dell servers, disk monitoring is a bit of a challenge. Dell’s OpenManage is a fine tool, but it cannot be run on non-RHEL versions of linux. So, how can one monitor health of hard drives when OpenManage is not an option?

An actual approach depends on the storage controller in use on the server.
Read the rest of this entry »


Tape changer script

Posted: November 30th, 2012 | Author: | Filed under: Backup | 4 Comments »

One of great features of GNU tar is that it can create multi-volume archives. This translates to ability of using multiple tapes when tar is used with a tape changer/library device. The only trick is to automate process of changing tapes. Such automation can be achieved by providing a script changing tape via “-F” argument to tar. Here, there is the script that I use with my changer (16x LTO5 tapes). Read the rest of this entry »


Linux – find information about hardware

Posted: November 19th, 2012 | Author: | Filed under: Linux | Comments Off on Linux – find information about hardware

Linux provides a great variety of tools to provide a wealth of information about hardware on which it is running.

dmidecode

Dmidecode is probably my favorite tool. It provides not only description of hardware but also other useful information such as serial numbers and BIOS version. To illustrate power of dmidecode, I was asked whether memory can be upgraded on one of our Dell servers. This includes finding out how many memory slots are available and how many are occupied at the present. All of this information can be obtained at once with dmidecode:

 dmidecode -t memory | grep Size
	Size: 16384 MB
	Size: 16384 MB
	Size: 16384 MB
	Size: 16384 MB
	Size: No Module Installed
	Size: No Module Installed
	Size: No Module Installed
	Size: No Module Installed
	Size: 16384 MB
	Size: 16384 MB
	Size: 16384 MB
	Size: 16384 MB
	Size: No Module Installed
	Size: No Module Installed
	Size: No Module Installed
	Size: No Module Installed

As the above shows, there are 16 memory slots on the server, with 8 of them holding 16GB DIMMs, and 8 being empty and available for upgrades. Read the rest of this entry »


Fedora 17 – “Cannot Display This Video Mode”

Posted: August 22nd, 2012 | Author: | Filed under: Linux | 2 Comments »

When attempting to install Fedora 17 on Sun Fire X4200 server, the following message displayed on the attached monitor:

Cannot Display This Video Mode

I tried a couple of different Dell monitors (both about 4-5 years old) with the same result. Fedora installation was available in text mode only. Unfortunately some of the options, such as custom disk partitioning, are available only during graphical (GUI) installation mode.

The following combination of boot parameters have done the trick to display GUI installation:

nomodeset xdriver=vesa resolution=800x600

Read the rest of this entry »