Skip to main content

Identifying virtual disks in Linux on vSphere

A default virtual machine has straightforward hardware. A single SCSI disk on a single SCSI card, for example. Having multiple SCSI disks or cards in a VM creates the need for in-guest identification. Linux complicates matters slightly by using alphabetical disk naming: /dev/sda, /dev/sdb, ... /dev/sdz, /dev/sdaa, /dev/sdab, ... This post looks at how you can identify individual disks in a VMware virtual machine.

Executive summary: VMware notation "X:Y" typically maps onto Linux scsi(X+2), Id:Y, which are then named in ascending order with /dev/sd* identifiers.

First step is to identify the SCSI cards:

The first two "scsi host" entries you'll see are the virtual IDE controllers, which Linux drives using its SCSI layer too. VMware uses these for one CD-ROM device (max four). The third "scsi host" called "host2" is the first virtual SCSI card. In the VM definition (the VMX file) you'd see this:

[root@testhost ~]# cd /sys/class/scsi_host
[root@testhost scsi_host]# ls -l
total 0
lrwxrwxrwx. 1 root root 0 May 22  2013 host0 -> ../../devices/pci0000:00/0000:00:07.1/host0/scsi_host/host0
lrwxrwxrwx. 1 root root 0 May 22  2013 host1 -> ../../devices/pci0000:00/0000:00:07.1/host1/scsi_host/host1
lrwxrwxrwx. 1 root root 0 May 22  2013 host2 -> ../../devices/pci0000:00/0000:00:10.0/host2/scsi_host/host2

Every device in /proc/scsi/scsi will be identified with the host that it's connected to. That, and the "Id" number, 

[root@testhost scsi_host]# cat /proc/scsi/scsi
Attached devices:
Host: scsi1 Channel: 00 Id: 00 Lun: 00
  Vendor: NECVMWar Model: VMware IDE CDR10 Rev: 1.00
  Type:   CD-ROM                           ANSI  SCSI revision: 05
Host: scsi2 Channel: 00 Id: 00 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0 
  Type:   Direct-Access                    ANSI  SCSI revision: 02

What VMware calls "0:0" is the disk on the first SCSI card (scsi2 in Linux), and "Id: 00". The default second disk would be virtually shown on "0:1", being scsi2 "Id: 01".

So in general, "SCSI disk X:Y" in VMware means scsi(X+2) ID: Y in Linux.

Linux names its disk devices at discovery time, during the boot cycle. So first the disks on scsi2 would be scanned, then scsi3, scsi4 and scsi5. On each scsi_host the disks will be added in ascending order, Id:00 first, up to 06, then 08 up to 15.

The only situation this could be different, is if you hot-add virtual disks or virtual SCSI cards without rebooting Linux. Both of these actions are supported in VMware, and by the Linux kernel. Those new cards and disks would be added to the list after the pre-existing ones, and no live device renaming/reordering happens for obvious reasons.

In a multiple SCSI card configuration you might notice that, while scsi2 (first virtual SCSI card) shows up on PCI bus 10, scsi3-5 show up behind a PCI bridge on PCI bus 11, together with the virtual network device(s).

On vSphere 5.5 and higher, both CD-ROM and disk devices can be shown on virtual SATA controllers, driven in Linux by the ahci driver. Each VMware virtual SATA controller (PCI ID 15ad:07e0 behind a PCI bridge on bus 11) will show up as 30 (!) scsi_host entries, and each can connect to one disk. The virtual disk number VMware shows, will be translated in the corresponding scsi_host number as reported by Linux. Device enumeration will still happen in ascending order over the scsi_hosts as before.


Grant said…
So while true if everything is provisioned initially....

The more reliable way to actually map this through is to pull the VM extended configuration data using powershell

$vm=vm-get "guest"
$vm.ExtensionData.Config.ExtraConfig | where {$_.Key -like "*pciSlotNumber")

Then follow the process described in this kb article

Howto Map PCI devices to guests

Once you have the bridge slot number convert it to hex and it's trivially mapped to the pci bridge in the /sys/pci path or lsscsi --verbose output.
Grant said…
So turns out that there's this quite elegant solution...

for DISK in $(ls /dev/sd*|grep -v [0-9]); do echo -n "$DISK "; udevadm info -n $DISK -a|grep label|cut -nc 20-24|tr "\n" ":"; udevadm info -n $DISK -a|grep 'KERNELS=="target'|cut -c 25; done

which works because the SCSI adapter is communicated to the guest in an attribute that's available to udev.

Bert de Bruijn said…
Thanks Grant, that's great info. I didn't think of udevadm info at the time. I tested this on CentOS 6 and 7, and on CentOS 6 there is no "label" attribute. The target line is there, but my kernels seem to count IDE controllers before lsilogic, and sata after pvscsi (the only combinations I have for immediate testing at the moment). The DRIVERS line shows which type of controller a device is on, which in the case of multiple different controllers is also very useful.

Popular posts from this blog

Volkswagen UHV bluetooth touch adapter & its problems

My Volkswagen car has the "universal cellphone preparation" UHV built-in. This is the main part of a car kit, but requires an additional adapter for connecting to a cellphone. At first, I was using an adapter for my good old Nokia 6310, even after I changed to the Nokia E71. Connecting was easy: pair the phone with the "VW UHV" bluetooth entity, and done. This has the phone connected to the car kit at all times, so even non-call-related functions use the car audio system (e.g. voice recognition).
But progress will have its way, no matter what happens. So in comes the "bluetooth touch adapter". Instead of a phone-specific adapter, this is a small touchscreen device that slots into the UHV dashboard mount. Connecting a phone is very different now:
the Bluetooth Touch Adapter connects to the "VW UHV" device via bluetooth
the phone connects to "Touch Adapter" device, also via bluetoothThe device doesn't allow step 2 if step 1 didn't s…

How to solve "user locked out due to failed logins" in vSphere vMA

In vSphere 6, if the vi-admin account get locked because of too many failed logins, and you don't have the root password of the appliance, you can reset the account(s) using these steps:

reboot the vMAfrom GRUB, "e"dit the entry"a"ppend init=/bin/bash"b"oot# pam_tally2 --user=vi-admin --reset# passwd vi-admin # Optional. Only if you want to change the password for vi-admin.# exitreset the vMAlog in with vi-admin These steps can be repeated for root or any other account that gets locked out.

If you do have root or vi-admin access, "sudo pam_tally2 --user=mylockeduser --reset" would do it, no reboot required.

Multiple VLANs on a Synology NAS

Synology, like other SOHO/SMB NAS vendors, touts VLAN functionality with their current DSM 4.1 software. However, the web interface just lets you specify one VLAN tag to use over each eth interface (or bond interface).
Manual approachIn the busybox environment that you can ssh into as root (after enabling ssh through the webinterface), there's all the tools you need to use multiple VLANs over one link (eth or bond), however:
First you insert the 802.1q module into the Linux kernel:
 /sbin/lsmod | /bin/grep -q 8021q || /sbin/insmod /lib/modules/8021q.koThen you add each VLAN you need to every interface (bond0 in this example)
 /sbin/vconfig add bond0 4And finally you can configure IP addresses on every interface.vlan combination (bond0.4 in this example)
 /sbin/ifconfig bond0.4 broadcast netmask same type of script would work on a QNAP NAS too, by the way. They offer 8021q.ko and vconfig in their commandline environment as well.
Packets from…