How (you might be able) to save a archive from the dreaded “premature end of file” !

Unless the archive is totally foobar one way is to use this quick command

pax -v -r * <  /path/to/archive

This will restore your archive to the paths in the archive (ie /home/user will be restored to that path) if you need different options man pax is your new friend.

Linux CLI on the FLY (interface stats)

I am going to start a new section of Linux CLI foo  that you may or may not know about. Many of these will be faster ways of doing things or getting information in a slightly different fashion. Many are old favorites of mine that I just wanted to share in case they help someone else.

Now without further ado the first command is :

ip -s l

short for ip stats list this gives a fast overview of your interfaces and  the types of packets that have been going through them. Output looks like this :

[root@pbnj lisa11]# ip -s l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
RX: bytes  packets  errors  dropped overrun mcast
1084866683 400954   0       0       0       0
TX: bytes  packets  errors  dropped carrier collsns
1084866683 400954   0       0       0       0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:30:48:61:c7:64 brd ff:ff:ff:ff:ff:ff
RX: bytes  packets  errors  dropped overrun mcast
552827144  1541759  0       0       0       0
TX: bytes  packets  errors  dropped carrier collsns
1428902993 1445402  0       0       0       0
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 00:30:48:61:c7:65 brd ff:ff:ff:ff:ff:ff
RX: bytes  packets  errors  dropped overrun mcast
0          0        0       0       0       0
TX: bytes  packets  errors  dropped carrier collsns
0          0        0       0       0       0

More information on interface stats can be found via:
ethtool -S $DEVICE
ifconfig $device (or just ifconfig for all devices)
ifconfig -s $DEVICE

among others….

Sluggish LVM/Disk/Raid performance under Linux (RHEL/CentOS 6)? Check power management!

If you are running tuned these settings will likely “undo” themselves. I found this out the hardway ;)

Update: If you are running a old firmware (older than CC34) update !! More on the update are on Seagates page and here http://niallbest.com/seagate-2tb-st32000542as-cc35-firmware-upgrade/. To find your firmware use `hdparm -I /dev/sd[a-z] | grep Firmware`.

I was noticing Samba being slow when I would access it from time to time. It would always work but man was it annoying. Since this was right after I added some new 2TB drives to my LVM/Software Raid 1 array I figured I would check to see if power management had been the cause for my pain. Sure enough I found this :

[root@stardust ~]# hdparm -C /dev/sd[a-z] | sed ‘/^$/d’
/dev/sda:
drive state is: active/idle
/dev/sdb:
drive state is: standby
/dev/sdc:
drive state is: active/idle
/dev/sdd:
drive state is: active/idle
/dev/sde:
drive state is: standby
/dev/sdf:
drive state is: active/idle
/dev/sdg:
drive state is: active/idle

To which I answered with :

hdparm -B 255 -S 0 -K 1 /dev/sd[b,e]

and wala all was happy again and my delay was gone!

/dev/sda:
drive state is: active/idle
/dev/sdb:
drive state is: active/idle
/dev/sdc:
drive state is: active/idle
/dev/sdd:
drive state is: active/idle
/dev/sde:
drive state is: active/idle
/dev/sdf:
drive state is: active/idle
/dev/sdg:
drive state is: active/idle

The new drives were Seagate “Green” drives (ST32000542AS) so I cant blame them for shipping with aggressive power save settings. If you need to check what your power save settings are you can do something like :

for d in /dev/sd[a-z]; do echo $d; hdparm -I $d | grep level; done

/dev/sda
/dev/sdb
Advanced power management level:  disabled
/dev/sdc
/dev/sdd
/dev/sde
Advanced power management level: disabled
/dev/sdf
/dev/sdg
HDIO_DRIVE_CMD(identify) failed: Invalid exchange

The /dev/sdg device is a USB drive and they dont play nice with hdparms commands usually.

This one image best sums the web server sysadmin part of my 10 years at a ISP.

Yet another (bad) use for the S3 amazon cloud

Found this URL on a website pop-up spam :

http://weeklycontestwinner.s3.amazonaws.com/***********

No I do not recommend clicking on it (but you might get something free …..yeah right). The interesting thing here is the use of the cloud for quick and easy  spammer/phishing/whatever-else-malware sites. Futher proof that :

A: Amazon isnt doing  good policing of content.
B: The Internet’s underbelly is light years ahead of most IT departments in understanding how to use the cloud.

I could keep going ..just wanted to share something I have seen growing in use over the last year.

You might have a issue if…

# uptime
15:05:59 up 271 days, 22:45,  2 users,  load average: 1789.01, 2333.21, 3025.28

This was a managed system where qmail ran away with a high remote concurrency set (5000). System was very responsive despite the load .

How DNS Siezures are like Chaos Theory

I am usually not one to get involved with politics and this matter is no different. Whichever way you lean on this DNS matter I feel it is a good chance to learn more about what options are out there. If you need to catchup here is some reading:

http://torrentfreak.com/bittorrent-based-dns-to-counter-us-domain-seizures-101130/
http://hotair.com/archives/2010/11/27/doj-seizes-domain-names-of-more-than-70-websites-suspected-of-piracy/

http://da.feedsportal.com/c/270/f/470440/s/1023e6ae
/l/0Lnews0Btechworld0N0Csme0C32510A10A0Ctorrent0Efinder0Eshut0Edown0Eby0Eus0Eagencies0C0Dolo0Frss/ia1.htm

These show how government is getting more involved in DNS which is the core of how we do things on the net. This opens the door for further understanding/learning of the other options that are available  on the net.

One of these is a great project for a open DNS system:

http://www.opennicproject.org

What OpenNIC are doing already and the push to a torrent driven DNS system with the .p2p extension are making the DNS world get interesting!

So what you have here is a change that is causing “ripples” all over the ‘net that could lead to major/minor changes into how DNS is done. Whatever you think will become of this it is going to be interesting to watch play out.

atop with a simulated high ICMP load

PRC | sys    0.22s  | user   0.01s  |               |               | #proc    158  | #zombie    0  | clones     0  |               |               | #exit      0  |
CPU | sys       1%  | user      0%  | irq     131%  |               | idle    668%  |               | wait      0%  |               | steal     0%  | guest     0%  |
cpu | sys       0%  | user      0%  | irq      80%  |               | idle     20%  |               | cpu007 w  0%  |               | steal     0%  | guest     0%  |
cpu | sys       0%  | user      0%  | irq      52%  |               | idle     48%  |               | cpu006 w  0%  |               | steal     0%  | guest     0%  |
cpu | sys       1%  | user      0%  | irq       0%  |               | idle     99%  |               | cpu004 w  0%  |               | steal     0%  | guest     0%  |
cpu | sys       0%  | user      0%  | irq       0%  |               | idle    100%  |               | cpu000 w  0%  |               | steal     0%  | guest     0%  |
CPL | avg1    0.07  | avg5    0.10  |               | avg15   0.07  |               | csw     3209  | intr   65623  |               |               | numcpu     8  |
MEM | tot     5.8G  | free    5.4G  | cache 185.4M  | dirty   0.0M  | buff   91.2M  | slab   33.8M  |               |               |               |               |
SWP | tot     2.2G  | free    2.2G  |               |               |               |               |               |               | vmcom 136.4M  | vmlim   5.1G  |
NET | transport     | tcpi       3  | tcpo       3  | udpi       0  | udpo       0  | tcpao      0  | tcppo      0  | tcprs      0  | tcpie      0  | udpip      0  |
NET | network       | ipi   438568  | ipo   438568  | ipfrw      0  | deliv 438569  |               |               |               | icmpi 438565  | icmpo 438565  |
NET | eth0     70%  | pcki  438531  | pcko  438566  | si   70 Mbps  | so   50 Mbps  | coll       0  | erri       0  | erro       0  | drpi       0  | drpo       0  |

PID    RUID         EUID          THR      SYSCPU      USRCPU      VGROW      RGROW     RDDSK      WRDSK     ST     EXC     S     CPUNR      CPU    CMD         1/1
24    root         root            1       0.18s       0.00s         0K         0K        0K         0K     –       -     S         7       6%    ksoftirqd/7
4314    root         root            1       0.01s       0.01s         0K         0K        0K         0K     –       -     S         0       1%    atop
3727    root         root            1       0.02s       0.00s         0K         0K        0K         0K     –       -     S         7       1%    kondemand/7
7825    root         root            1       0.01s       0.00s         0K         0K        0K         0K     –       -     R         4       0%    atop
7135    root         root            1       0.00s       0.00s         0K         0K        0K         0K     –       -     S         7       0%    sshd

This was created with several `ping -s 1 -q -f $IP` commands across several systems. The CPU is :

Intel(R) Xeon(R) CPU W3530  @ 2.80GHz

What can I say , I am impressed ! I am going to try and max it out and see what it can do all out! This is a HUGE jump from our old P4 based firewall that barfed at 80-100K pps inbound!

Geek Ghetto – The Pen


Customer had started a fsck and didn’t set it to auto fix issues. Got into the check and , after several minutes, found the need to hold the “Y” key permanently. Since he was ready to eat lunch I hooked him up with some ghetto engineering.

Magento Enterprise 1.9.0.0 Worker vs Prefork MPM

Had to do a quick test of a Magento server getting ready for production recently. Being a big fan of Apaches Worker MPM (even when using PHP with ZTS) I thought i would test their recommended settings for Prefork vs one of my homebrew Worker configs. The results can be found on my wiki at  http://misterx.org/wiki/index.php/Worker_vs_prefork_MPM.