What's a “drive,” or “volume?”

July 10, 2020

When you set up your library, you’ll see this option:

PhotoStructure uses the words “disk,” “drive,” and “volume” to mean any storage device available to your computer.

This includes:

The internal hard drive of your laptop or desktop
An external USB hard drive or thumb drive
An SD card in a built-in or external card reader
A shared network directory from your NAS, possibly spanning several drives in a RAID.

For the more precise—nay, pedantic—readers: technically, disks are physical data storage, and volumes are logical storage. Single volumes may use just a portion of one disk, or partitions across several disks.

Just like macOS users are used to the term “folder,” and Windows users use the term “directory,” these terms have been muddied as well. Windows users are used to mounting “disks” to “drive letters”. MacOS mounts to /volumes. As PhotoStructure is cross-platform, no term will be “native” to both sets of users.

fascinating. really. #

I’m getting to the good bit, hang on. It’ll be interesting, I swear.

…

Fine. Mildly interesting.

…

PhotoStructure needs a stable way to refer to the volumes you’ve mounted to your system.

Why?

If you plug in a new external hard drive into a Mac, it will mount to /Volumes/Untitled.

If you plug in another new hard drive, or untitled SD card, that will be mounted to /Volumes/Untitled 1.

Reboot, or unplug and re-plug those devices, and which device gets Untitled vs Untitled 1 is random, and can swap. Or be given Untitled 2.

Why is that bad? #

Earlier versions of PhotoStructure would see these swapped volume mountpoints as new volumes, and re-import that volume. This could potentially double the size of your library’s database. The files would all be deduplicated, but larger databases consume more RAM, and are slower.

And Windows and Ubuntu users, don’t get smug: Windows is happy to give random drive letters to external devices, and Ubuntu uses similar mounting heuristics to macOS.

I smell a buildup to some clever solution… #

Old versions of PhotoStructure described files in your library database by using the full pathname to your photo or video, so D:\\Pictures\\image.jpg would be file:///D:/Pictures/image.jpg.

We want PhotoStructure to provide the freedom to read and write your library on multiple computers, even when they’re different operating systems, or using PhotoStructure for Desktops or PhotoStructure for Servers. Your library shouldn’t be locked to an OS or hardware.

That portability requirement means that /D:/ part of the URI isn’t going to work (even if it was stable), because that drive letter is a windows-specific thing. We need a better way to persistently identify volumes across operating systems.

Volume UUIDs save the day… #

To get this persistent volume identifier, PhotoStructure asks the operating system for the “logical volume UUID” of each volume. These looks something like 02230e7a-c32f-11ea-92b5-1bf036f2d97b.

PhotoStructure takes a short SHA of that UUID, like 3DNCoKQz4, instead of the volume’s mountpoint or drive letter, to describe the file in your PhotoStructure library database. The URI in your database then looks like psfile://3DNCoKQz4/Pictures/image.jpg.

This solves both our problems: if the drive letter changes, the URI doesn’t change, and if you plug in that device into, say, your Mac or Linux box, we still get the same URI. 🎉

…except when they don’t #

Almost immediately after getting a shoulder cramp from the gosh-that’s-so-clever self-congratulations, our parade got rained out, because logical volume UUIDs aren’t always available.

Volumes exported into a docker containers don’t have UUIDs.
Network filesystems (like an AFS or SMB mount from a NAS) don’t expose a UUID.
Some physical storage devices (like thumb drives and even some external hard drives) don’t have the hardware controller support that exposes a UUID.

The workaround… #

To work around this issue, PhotoStructure will use either the logical volume UUID, or generate a new random UUID, and try to write it into a .uuid text file stored in the root directory of each volume.

This allows the volume to maintain a consistent UUID when mounted on a different device (which may not expose the UUID).

Note that the .uuid file overrides the logical volume UUID.

If you’re using Docker, or importing files from a read-only device, like your NAS, PhotoStructure may not have sufficient permissions to write to the root directory to write this .uuid file.

How to manually add `.uuid` files #

For volumes that you don’t want PhotoStructure to have write access to, you can create this file manually so your library can still get the benefit of stable URIs.

On Debian or Ubuntu:

# To set the uuid to match the hardware uuid, run `lsblk -P --output mountpoint,uuid`, 
# find the appropriate UUID, and cat the contents into $VOLUME_MOUNTPOINT/.uuid

# Not all hardware devices have a UUID, though. In that case, run the following:

# Install the `uuid` command:
sudo apt install uuid

# set $VOLUME_MOUNTPOINT to the volume you want to give a UUID to:
sudo uuid -o $VOLUME_MOUNTPOINT/.uuid

# Make sure PhotoStructure can read the contents:
sudo chmod ugo+r $VOLUME_MOUNTPOINT/.uuid

On macOS (with homebrew):

brew install ossp-uuid

# set $VOLUME_MOUNTPOINT to the volume you want to give a UUID to:
sudo uuid -o $VOLUME_MOUNTPOINT/.uuid

# Make sure PhotoStructure can read the contents:
sudo chmod ugo+r $VOLUME_MOUNTPOINT/.uuid

On Windows, in PowerShell

# (replace `F:` with the proper drive letter):
New-Guid | Select-Object -ExpandProperty Guid | Out-File -NoClobber -FilePath F:\.uuid

# (There isn't a Set-Acl command here, as I'm assuming that the user you run
# PhotoStructure with can already read the new .uuid file. If this isn't the case,
# please add the read permission to this file.)

Note that this UUID doesn’t actually need to match the physical volume UUID. If there’s a .uuid file in the root of a device, the file’s contents will be used instead of the physical volume UUID.

Some caveats about Volume UUIDs #

If you add a new `.uuid` file, or change the contents of a `.uuid` file: #

You’ll need to run rebuild via the system navigation menu to have your URIs be re-created.

You can select rebuild while an import is running: it will fix currently-imported assets, and then continue with importing.

UUIDs must be unique! #

If you clone a disk that has a .uuid file, make sure you write a different, new .uuid file to the cloned disk. Use the instructions above specific to your OS.

If 2 different physical devices share the same .uuid value, PhotoStructure will assume they are the same physical device. If contents are different between the devices, the last-scanned volume will “win” and library metadata for that volume will overwrite the metadata from the other device.

Note that your photos and videos won’t be altered on either device, but your library will only reflect the contents of the last-scanned volume with that UUID.

Volume UUIDs aren’t a panacea #

Volume UUIDs only address duplicates that arise from

the same directory being mounted to different directories
the same directory being mounted on different computers

If you mount a subdirectory of a volume, PhotoStructure can’t detect this situation, and will re-import the assets it finds in that subdirectory.

This situation isn’t theoretical: if you mount a remote SMB filesystem, there will typically be two paths to your “home directory”: \\server\\home, and \\server\homes\username.

What this results in is two paths that point to the same directory. PhotoStructure doesn’t have heuristics to detect this situation, and will think that the contents of \\nas\alice and \\nas\homes\alice are different directories, and import both directories.

All assets from one volume will be marked as a duplicate of the other volume, so your asset count won’t increase, but your asset file count will double. (see “Library Metrics” for more details).

FAQ
Docs

Hi! I’m Matthew, the author of PhotoStructure. I’m a dad, amateur photographer, and author of open source, adtech, fintech, and edtech software since the turn of the millennium. He/him. I am building a self-hosted, safe, and fast new home for all your photos and videos.
Your memories deserve PhotoStructure. Try it out for free!