the birdhouse in my soul

A customer asked me to verify their vSphere implementation. Everything looked perfectly redundant, in the traditional elegant way: cross over between layers to avoid single points of failure. I had to break the bad news: too much redundancy can mean NO redundancy . In this case: host has 4 network interfaces (2x dual port card). VM's connect to a vSwitch, which has redundancy over vmnic0 and vmnic2 (using 1 port of each card). Another vSwitch for the storage traffic, same level of redundancy, using vmnic1 and vmnic3. Looking good. Then the physical level. 4 host interfaces, 2 interconnected network switches. The traditional |X| design connects the two interfaces of every card to different switches. Looking good. But looking at both configurations together, you'll see that every vSwitch gets connected to one physical switch. The sum of two crossed redundancy configurations equals no redundancy at all. Enabling CDP or LLDP can help you identify this problem, as you can identi...

Found out the hard way: don't use underscores in hostnames. It's not allowed by DNS, and it breaks things. In this case: joining vCenter Server Appliance (VCSA) in an Active Directory doesn't work if the hostname of the appliance contains an underscore (_). It also doesn't work if the hostname is "localhost". If your appliance uses DHCP, the appliance gets its hostname through reverse DNS. So in that case, it _is_ a freaking DNS problem.

I won't blog about the whole vhv.allow="true" procedure here, that's been covered elsewhere. But what does nested virtualization change in a VM ? Well, the CPU features that are exposed change: A regular 64-bit Linux VM sees # grep flags /proc/cpuinfo flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc up arch_perfmon pebs bts rep_good xtopology tsc_reliable nonstop_tsc aperfmperf unfair_spinlock pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx hypervisor lahf_lm ida arat A 64-bit VM with nested virtualization enabled sees # grep flags /proc/cpuinfo flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc up arch_perfmon pebs bts rep_good xtopology tsc_reliable nonstop_tsc aperfmperf unfair_spinlock pni pclmulqdq vmx ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx hy...

When setting up backups over SSH (e.g. rsnapshot with rsync over SSH), it's important to know that the default SSH cipher isn't necessarily the fastest one. In this case, the CPU-based encryption is the performance bottleneck, and making it faster means getting faster backups. A test (copying a 440 MB file between a fast Xeon CPU (fast=no bottleneck there) and an Atom based NAS) shows that the arcfour family of ciphers are clearly the fastest in this setup: cipher real time user time bandwidth arcfour 0m9.639s 0m7.423s 45.7 MB/s arcfour128 0m9.751s 0m7.483s 45.1 MB/s arcfour256 0m9.856s 0m7.764s 44.7 MB/s blowfish-cbc 0m13.093s 0m10.909s 33.6 MB/s aes128-cbc 0m22.565s 0m20.129s 19.5 MB/s aes128-ctr 0m25.400s 0m22.951s 17.3 MB/s aes192-ctr 0m28.047s 0m25.771s 15.7 MB/s 3des-cbc 0m51.067s 0m48.018s 8.6 MB/s The default configuration of openssh uses aes128-ctr, so changing the cipher to arcfour gets me a 2.5-fold increase in bandwidth here ! Use the "Ciph...

My VI3 and vSphere4 home lab consisted of whitebox PCs. For VI3 I used MSI based nonames, for vSphere4 I used Shuttle SX58j3. For the new vSphere5 generation, I wanted some real server hardware. Because of shallow depth requirements, the choice of rackmount servers was limited. I picked the Dell Poweredge R210II instead of the sx58j3 because - on the vSphere HCL (the sx58j3's won't boot vSphere5 RC !) - Sandy Bridge low TDP CPUs available (I got the E3-1270) - onboard dual BCM5716 nics support iSCSI offload (aka "dependent HW iSCSI") - IPMI built-in (not tested yet) - dense: 1U (the sx58j3 is about 4 units, but can fit 2 in 19") - one free PCIe slot (The sx58j3 has 2 slots, but needs a VGA card) - not incredibly expensive (up to 16GB RAM) Downsides: - only one free PCIe slot (max GbE nics needs expensive quadport card) - incredibly expensive (with 32GB RAM it's 3x the price of a 16GB config) - can't buy without at least one disk. I'll be ru...

At a customer, a Linux workstation suddenly refused to open HTTPS sites. Verified recent package versions of both browser (konqueror) and libraries (kde, openssl), everything looked good, but it didn't work. This blogpost serves as documentation for the fact that checking new software isn't enough, because in this case removing old openssl compatibility libraries solved the problem. The kio_http helper is not linked with openssl directly, and for some reason it must have tried to open one of the old openssl versions that were also installed. After erasing all versions between 0.9.5a and 0.9.6b, keeping the current 0.9.8e, konqueror had no problems opening https sites anymore.

My home lab got upgraded with a new gigabit switch recently. Main improvement I wanted over the old Linksys SLM2024 I had: Cisco Discovery Protocol. Based on that requirement and the budget, I selected the Cisco SG300-28 Small Business managed switch. The web interface is clearly improved compared to the SLM2024, and CDP is a real treat. Both vSphere ESXi and the cdpr utility under Linux decode the CDP information nicely. CDP is a great help to find errors in patch cable arrangement !