synaesthesia

So you’ve a got a disk in your hand that’s new, used, or questionably refurbished, and you want to give it a thorough thrashing to see how it holds up before trusting your data to it. How? SMART tests and badblocks are the usual tools, but you really want to simulate a workload instead of just doing a linear once-over.

Fortunately, the fio benchmarking tool has a randrw module that can do exactly this. Run this and it will scribble all over a disk and check the results. You should then have a good idea whether that disk is a winner or a dud for a real world workload.

Example, assuming we will stress-test the disk for 1 day (86400 seconds), we have 4 CPU threads, and the disk to be tested is /dev/sdg (WARNING: all data on this disk will be randomly trashed):

# fio --name=randrw --time_based --runtime=86400 --ioengine=libaio --iodepth=64 --rw=randrw --bs=4k --direct=1 --numjobs=4 --filename=/dev/sdg

Enabling SSD TRIM support on Linux can be interesting due to the many storage layers involved. TRIM must be enabled by enabling the relevant discard option at least at the following levels (if they exist in a configuration):

• mdraid (manually in the case of RAID4/5/6)
• dm-crypt
• LVM

Additionally, the filesystem must either be configured with the discard to issue TRIM commands automatically after filesystem operations that clear blocks, or a scheduled fstrim must be performed during a maintenance window. With frequently LVM operations, discard-everything may be a better choice.

But generally, if a device supports queued TRIM and the implementation is not broken in the SSD firmware, then enabling at the filesystem level as well should be a reasonable approach.

To set up a new SSD in Linux, generally Debian’s SSD Optimization wiki is a good place to start.

Here are some questions about interesting TRIM scenarios that I determined the answers to.

Linux hybrid mdraid, mixing SSD and HDD devices

Can a hybrid RAID be trimmed by the filesystem running on top of it? The long and short answer to this question is: actually, this works fine. The TRIM command is sent only to hardware devices which advertise support for it, so it won’t be sent to the HDD. If trimming the filesystem causes errors, check for TRIM support in other intermediate layers (LVM, dm-crypt, mdraid).

Linux software RAID4/5/6

The raid456 driver cannot apparently automatically query the underlying storage layers to determine whether the device zeroes data blocks that have been discarded, and failing to do this is considered unsafe.

So one must check manually the SSD devices using lsblk -D for this feature, which is blacklisted for some buggy hardware:

valhalla:~# lsblk -D
NAME                  DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
sda                          0      512B       2G         1
??sda1                       0      512B       2G         1
??sda2                       0      512B       2G         1
? ??md0                      0      512B       2G         1
??sda3                       0      512B       2G         1

If the RAID underlying devices (check /proc/mdstat) all have ones in the DISC-ZERO column, it is safe to enable the module parameter:

parm:           devices_handle_discard_safely:Set to Y if all devices in each array reliably return zeroes on reads from discarded regions (bool)

Which is done by adding raid456.devices_handle_discard_safely=1 to one’s kernel command line in the bootloader configuration.

TRIMming all free space on a disk

Once everything is all set up, one may want to TRIM any unused space in case it was previously written to for whatever reason. This is easy to do as long as an LVM physical volume exists over the entire disk. Simply create a logical volume that uses up all the free space, then delete that volume:

lvcreate -l100%FREE -n blkdiscard SSD-VG
lvremove SSD-VG/blkdiscard

I encountered a bit of trouble using an old Creative Webcam 3 USB with a website that wanted to capture video from it.  This camera uses an OmniVision OV511+ controller and had never been any trouble to use even with Linux.  However, attempting to use it via the browser produced a message that access to the camera was denied.

The first thing to check was permissions on the /dev/video0 device. I found that the user was not in the video group that was required to access the device by default. However, granting this permission and restarting the login session was still not enough.

It turns out that a characteristic of the USB Video Class (UVC) standard is that only video frames with certain pixel encodings can be generated by compliant devices. Whether due to this narrow scope or for other reasons, the WebRTC standard, which provides access to cameras via HTML5-compliant web browsers, only incorporated support for a relatively small number of pixel encodings, and browsers using the WebRTC library therefore only implement support for, at most, that subset of possible pixel encodings.

For example, Chrome (and any other browser which uses the WebRTC library) only supports decoding just a handful of the dozens of raw pixel formats, many vendor-specific, that are supported by the Video4Linux2 API.

Unfortunately, one of those vendor-specific pixel formats that is not supported by the WebRTC layer is the O511 pixel format generated by the OV511+ chip in the Creative Webcam 3 USB. This can be confirmed like so:

$ lsusb
Bus 001 Device 010: ID 05a9:a511 OmniVision Technologies, Inc. OV511+ Webcam
$ v4l2-ctl --all -d /dev/video0
[..]
Format Video Capture:
   Width/Height  : 640/480
   Pixel Format  : 'O511'

Okay, so what can be done? Fortunately, V4L2 developers provided a compatibility wrapper that will convert frames from esoteric pixel formats on-the-fly to the much more widely supported BGR42/YUV420 pixel formats. The wrapper is loaded using LD_PRELOAD before the browser is launched, e.g:

$ LD_PRELOAD=/usr/lib/libv4l/v4l2convert.so firefox
$ LD_PRELOAD=/usr/lib/libv4l/v4l2convert.so chromium-browser

If your distribution is multiarch, the library will be under /usr/lib/x86_64-linux-gnu or similar.