Skip to main content

Identifying virtual disks in Linux on vSphere

A default virtual machine has straightforward hardware. A single SCSI disk on a single SCSI card, for example. Having multiple SCSI disks or cards in a VM creates the need for in-guest identification. Linux complicates matters slightly by using alphabetical disk naming: /dev/sda, /dev/sdb, ... /dev/sdz, /dev/sdaa, /dev/sdab, ... This post looks at how you can identify individual disks in a VMware virtual machine.

Executive summary: VMware notation "X:Y" typically maps onto Linux scsi(X+2), Id:Y, which are then named in ascending order with /dev/sd* identifiers.


First step is to identify the SCSI cards:

The first two "scsi host" entries you'll see are the virtual IDE controllers, which Linux drives using its SCSI layer too. VMware uses these for one CD-ROM device (max four). The third "scsi host" called "host2" is the first virtual SCSI card. In the VM definition (the VMX file) you'd see this:

[root@testhost ~]# cd /sys/class/scsi_host
[root@testhost scsi_host]# ls -l
total 0
lrwxrwxrwx. 1 root root 0 May 22  2013 host0 -> ../../devices/pci0000:00/0000:00:07.1/host0/scsi_host/host0
lrwxrwxrwx. 1 root root 0 May 22  2013 host1 -> ../../devices/pci0000:00/0000:00:07.1/host1/scsi_host/host1
lrwxrwxrwx. 1 root root 0 May 22  2013 host2 -> ../../devices/pci0000:00/0000:00:10.0/host2/scsi_host/host2

 Every device in /proc/scsi/scsi will be identified with the host that it's connected to. That, and the "Id" number, 

 [root@testhost scsi_host]# cat /proc/scsi/scsi
Attached devices:
Host: scsi1 Channel: 00 Id: 00 Lun: 00
  Vendor: NECVMWar Model: VMware IDE CDR10 Rev: 1.00
  Type:   CD-ROM                           ANSI  SCSI revision: 05
Host: scsi2 Channel: 00 Id: 00 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 1.0 
  Type:   Direct-Access                    ANSI  SCSI revision: 02

What VMware calls "0:0" is the disk on the first SCSI card (scsi2 in Linux), and "Id: 00". The default second disk would be virtually shown on "0:1", being scsi2 "Id: 01".

So in general, "SCSI disk X:Y" in VMware means scsi(X+2) ID: Y in Linux.

Linux names its disk devices at discovery time, during the boot cycle. So first the disks on scsi2 would be scanned, then scsi3, scsi4 and scsi5. On each scsi_host the disks will be added in ascending order, Id:00 first, up to 06, then 08 up to 15.

The only situation this could be different, is if you hot-add virtual disks or virtual SCSI cards without rebooting Linux. Both of these actions are supported in VMware, and by the Linux kernel. Those new cards and disks would be added to the list after the pre-existing ones, and no live device renaming/reordering happens for obvious reasons.

In a multiple SCSI card configuration you might notice that, while scsi2 (first virtual SCSI card) shows up on PCI bus 10, scsi3-5 show up behind a PCI bridge on PCI bus 11, together with the virtual network device(s).

On vSphere 5.5 and higher, both CD-ROM and disk devices can be shown on virtual SATA controllers, driven in Linux by the ahci driver. Each VMware virtual SATA controller (PCI ID 15ad:07e0 behind a PCI bridge on bus 11) will show up as 30 (!) scsi_host entries, and each can connect to one disk. The virtual disk number VMware shows, will be translated in the corresponding scsi_host number as reported by Linux. Device enumeration will still happen in ascending order over the scsi_hosts as before.


Comments

Grant said…
So while true if everything is provisioned initially....

The more reliable way to actually map this through is to pull the VM extended configuration data using powershell

$vm=vm-get "guest"
$vm.ExtensionData.Config.ExtraConfig | where {$_.Key -like "*pciSlotNumber")

Then follow the process described in this kb article

Howto Map PCI devices to guests

Once you have the bridge slot number convert it to hex and it's trivially mapped to the pci bridge in the /sys/pci path or lsscsi --verbose output.
8/6/16 07:37
Grant said…
So turns out that there's this quite elegant solution...


for DISK in $(ls /dev/sd*|grep -v [0-9]); do echo -n "$DISK "; udevadm info -n $DISK -a|grep label|cut -nc 20-24|tr "\n" ":"; udevadm info -n $DISK -a|grep 'KERNELS=="target'|cut -c 25; done



which works because the SCSI adapter is communicated to the guest in an attribute that's available to udev.

8/6/16 09:13
Bert de Bruijn said…
Thanks Grant, that's great info. I didn't think of udevadm info at the time. I tested this on CentOS 6 and 7, and on CentOS 6 there is no "label" attribute. The target line is there, but my kernels seem to count IDE controllers before lsilogic, and sata after pvscsi (the only combinations I have for immediate testing at the moment). The DRIVERS line shows which type of controller a device is on, which in the case of multiple different controllers is also very useful.
8/6/16 10:32
Post a Comment

Popular posts from this blog

Volkswagen UHV bluetooth touch adapter & its problems

My Volkswagen car has the "universal cellphone preparation" UHV built-in. This is the main part of a car kit, but requires an additional adapter for connecting to a cellphone. At first, I was using an adapter for my good old Nokia 6310, even after I changed to the Nokia E71. Connecting was easy: pair the phone with the "VW UHV" bluetooth entity, and done. This has the phone connected to the car kit at all times, so even non-call-related functions use the car audio system (e.g. voice recognition). But progress will have its way, no matter what happens. So in comes the "bluetooth touch adapter". Instead of a phone-specific adapter, this is a small touchscreen device that slots into the UHV dashboard mount. Connecting a phone is very different now: the Bluetooth Touch Adapter connects to the "VW UHV" device via bluetooth the phone connects to "Touch Adapter" device, also via bluetooth The device doesn't allow step 2 if step 1 didn'
12 comments
Read more

Reset lost root password on vSphere ESXi 6.7

VMware's solution to a lost or forgotten root password for ESXi is simple: go to  https://kb.vmware.com/s/article/1317898?lang=en_US  and you'll find that "Reinstalling the ESXi host is the only supported way to reset a password on ESXi". If your host is still connected to vCenter, you may be able to use Host Profiles to reset the root password, or alternatively you can join ESXi in Active Directory via vCenter, and log in with a user in the "ESX Admins" AD group. If your host is no longer connected to vCenter, those options are closed. Can you avoid reinstallation? Fortunately, you can. You will need to reset and reboot your ESXi though. If you're ready for an unsupported deep dive into the bowels of ESXi, follow these steps: Create a bootable Linux USB-drive (or something else you can boot your server with). I used a CentOS 7 installation USB-drive that I could use to boot into rescue mode. Reset your ESXi and boot from the Linux medium. Ident
Post a Comment
Read more

GEM WS2 MIDI System Exclusive structure and checksums

MIDI is the standard for communication between electronic music instruments like keyboards and synthesizers. And computers! While tinkering with an old floppy-less GEM WS2 keyboard, I wanted to figure out the structure of their System Exclusive memory dumps. SysEx is the vendor-specific (and non-standard) part of MIDI. Vendors can use it for real-time instructions (changing a sound parameter in real-time) and for non-real-time instructions (sending or loading a configuration, sample set, etc.). In the GEM WS2, there's two ways of saving the memory (voices, globals, styles and songs): in .ALL files on floppy, and via MIDI SysEx. The .ALL files are binary files, 60415 bytes long. The only recognizable parts are the ASCII encoded voice and global names. The SysEx dumps are 73691 bytes long. As always in MIDI, only command start (and end) bytes have MSB 1, and all data bytes have MSB 0. The data is spread out over 576 SysEx packets, preceded by one SysEx packet with header informat
23 comments
Read more