What's a “drive,” or “volume?”

When you set up your library, you’ll see this option:

PhotoStructure uses the words “disk,” “drive,” and “volume” to mean any storage device available to your computer.

This includes:

  • The internal hard drive of your laptop or desktop
  • An external USB hard drive or thumb drive
  • An SD card in a built-in or external card reader
  • A shared network directory from your NAS, possibly spanning several drives in a RAID.

For the more precise—nay, pedantic—readers: technically, disks are physical data storage, and volumes are logical storage. Single volumes may use just a portion of one disk, or partitions across several disks.

Just like macOS users are used to the term “folder,” and Windows users use the term “directory,” these terms have been muddied as well. Windows users are used to mounting “disks” to “drive letters”. MacOS mounts to /volumes. As PhotoStructure is cross-platform, no term will be “native” to both sets of users.

💽 Why does PhotoStructure need to know about volumes? 🔗

PhotoStructure needs a stable way to refer to your photos and videos.

If you plug in a new external hard drive into a Mac, it will mount to /Volumes/Untitled, and PhotoStructure may import /Volumes/Untitled/photo.jpg from that directory.

If you plug in another new hard drive, or untitled SD card, that will be mounted to /Volumes/Untitled 1. Let’s say PhotoStructure then imports /Volumes/Untitled 1/photo.jpg.

If you reboot, or unplug and re-plug those devices, macOS will give a random device the Untitled vs Untitled 1 mountpoint.

/Volumes/Untitled 1/photo.jpg could point to either file. PhotoStructure’s sync will figure out that the file changed, and update the database and preview images, but if that disk has 100,000 images, that’s a lot of work to do every time you plug in a drive.

Earlier versions of PhotoStructure would see these swapped volume mountpoints as new volumes, and re-import that volume. This could potentially double the size of your library’s database. The files would all be deduplicated, but larger databases consume more RAM, and are slower.

And Windows and Ubuntu users, don’t get smug: Windows is happy to give random drive letters to external devices, and Ubuntu uses similar mounting heuristics to macOS.

🤔 Is there a better way to point to files? 🔗

Old versions of PhotoStructure described files in your library database by using the full pathname to your photo or video, so D:\\Pictures\\image.jpg would be file:///D:/Pictures/image.jpg.

We want PhotoStructure to provide the freedom to read and write your library on multiple computers, even when they’re different operating systems, or using PhotoStructure for Desktops or PhotoStructure for Servers. Your library shouldn’t be locked to an OS or hardware.

That portability requirement means that /D:/ part of the URI isn’t going to work (even if it was stable), because that drive letter is a windows-specific thing. We need a better way to persistently identify volumes that works across macOS, Linux, and Windows.

🤓 Logical Volume UUIDs! 🔗

macOS, Linux, and Windows all provide access to a “logical volume UUID.” On Linux, this is exposed via lsblk, on macOS, diskutil, and Windows, Get-Volume. These UUIDs look something like 02230e7a-c32f-11ea-92b5-1bf036f2d97b.

This looks like exactly what we were looking for: the UUIDs are the same across the different platforms, and don’t change unless you reformat the volume.

PhotoStructure takes a short SHA of that UUID, like 3DNCoKQz4, and substitutes this short SHA for the file’s volume mountpoint path. The URI in your database then looks like psfile://3DNCoKQz4/Pictures/image.jpg.

This seems like we’re done! The URI won’t change if the drive letter or mountpoint changes, and if you plug in that same disk into, say, your Mac or Linux box, we still get the same URI. 🎉

💔 …except when they don’t 🔗

Unfortunately, logical volume UUIDs aren’t available on all devices and volumes.

  • Subdirectories exported into docker containers don’t expose logical volume UUIDs (and even if they did, the bind mount could be at an arbitrary subdirectory).

  • Network filesystems (like an AFS or SMB mount from a NAS) don’t have an API to access logical UUIDs.

  • Some physical storage devices (like thumb drives and even some cheap external hard drives) don’t have hardware controller support for volume UUIDs.

🪛 The workaround… 🔗

To work around this issue, PhotoStructure uses the following heuristics:

  1. If a volume’s root directory contains a .uuid file, assume the contents is a valid volume UUID and use that.

  2. If the .uuid file is missing, try to fetch the logical volume UUID. If that fails, generate a new random UUID. Attempt to write it into a .uuid text file stored in the root directory of the volume.

These heuristics allow the volume to maintain a consistent UUID when mounted on a different device or exposed via remote filesharing.

Note that the contents of a .uuid file will override the logical volume UUID.

If you’re using Docker, or importing files from a read-only device, like your NAS, PhotoStructure may not have sufficient permissions to write to the root directory to write this .uuid file, but you can help out and add it yourself.

✍️ How to manually add .uuid files 🔗

For volumes that you don’t want PhotoStructure to have write access to, you can create this file manually so your library can still get the benefit of stable URIs.

On POSIX systems that have Node.js installed:

node -p "require('crypto').randomUUID()" | UMASK=0022 sudo tee $VOLUME_MOUNTPOINT/.uuid

On Linux:

# To set the uuid to match the hardware uuid, run `lsblk -P --output mountpoint,uuid`,
# find the appropriate UUID, and cat the contents into $VOLUME_MOUNTPOINT/.uuid

# Not all hardware devices have a UUID, though. In that case, run the following:

# Install the `uuid` command (for Debian/Ubuntu/Mint)--for other distributions,
# run the appropriate command to install the "uuid" package from http://www.ossp.org/pkg/lib/uuid/
sudo apt install uuid

# set $VOLUME_MOUNTPOINT to the volume you want to give a UUID to:
sudo uuid -o $VOLUME_MOUNTPOINT/.uuid

# Make sure PhotoStructure can read the contents:
sudo chmod ugo+r $VOLUME_MOUNTPOINT/.uuid

On macOS with uuidgen (for macOS 10.12+):

# set $VOLUME_MOUNTPOINT to the volume you want to give a UUID to:
uuidgen | sudo tee $VOLUME_MOUNTPOINT/.uuid

# Make sure PhotoStructure can read the contents:
sudo chmod ugo+r $VOLUME_MOUNTPOINT/.uuid

On older macOS (with homebrew):

brew install ossp-uuid

# set $VOLUME_MOUNTPOINT to the volume you want to give a UUID to:
sudo uuid -o $VOLUME_MOUNTPOINT/.uuid

# Make sure PhotoStructure can read the contents:
sudo chmod ugo+r $VOLUME_MOUNTPOINT/.uuid

On Windows, in PowerShell

# (replace `F:` with the proper drive letter):
New-Guid | Select-Object -ExpandProperty Guid | Out-File -NoClobber -FilePath F:\.uuid

# (There isn't a Set-Acl command here, as I'm assuming that the user you run
# PhotoStructure with will be able to read this new .uuid file. If this isn't the case,
# please add the proper read permissions to this file.)

Note that this UUID doesn’t actually need to match the physical volume UUID. If there’s a .uuid file in the root of a device, the file’s contents will be used instead of the physical volume UUID.

🏗️ If you add or change a .uuid 🔗

PhotoStructure assumes that volume UUIDs do not change.

If you change the contents of a .uuid or add a new .uuid to a remote volume you must rebuild your library, via the system navigation menu.

Read more about rebuilds here.

You can select rebuild while an import is running: it will fix currently-imported assets, and then continue with importing.

🦄 Important: UUIDs must be unique! 🔗

If you clone a disk that has a .uuid file, make sure you write a different, new .uuid file to the cloned disk. Use the instructions above specific to your OS.

If 2 different physical devices share the same .uuid value, PhotoStructure will assume they are the same physical device. If contents are different between the devices, the last-scanned volume will “win” and library metadata for that volume will overwrite the metadata from the other device.

Note that your photos and videos won’t be altered on either device, but your library database will only reflect the contents of the last-scanned volume with that UUID.

🤓 Don’t you mean UID, not UUID? 🔗

Technically, these are UIDs, not UUIDs.

The terms UUID and UID aren’t properly interchangeable, but the difference is subtle. UUIDs only have a handful of official accepted versions, and a UID, or unique identifier can be anything, as long it is unique for the set they are applicable to.

Unfortunately, some hard drive manufacturers return physical volume IDs that are not valid UUIDs, as they are only 12 or 16 characters long. To work around this, PhotoStructure’s volume UUID validation is very lax, and accepts any string matching the regular expression /[0-9a-z-]{8,}/ as a valid volume UUID.

😐 Volume UUIDs don’t fix everything 🔗

Volume UUIDs only address duplicates that arise from

  1. the same directory being mounted to different mountpoints
  2. the same directory being mounted on different computers

PhotoStructure can’t detect if you mount a subdirectory of a volume: it will re-import the assets it finds in that subdirectory.

This situation isn’t theoretical: if you mount a remote SMB filesystem, there will typically be two paths to your “home directory”: \\server\\home, and \\server\homes\username.

In this situation, two URIs will point to the same directory. PhotoStructure can’t detect this situation and will think that the contents of \\nas\alice and \\nas\homes\alice are different directories and import both directories.

All assets from one volume will be marked as a duplicate of the other volume, so your asset count won’t increase, but your asset file count will double. (see “Library Metrics” for more details).

🙅 How do I disable .uuid writing? 🔗

Set the system setting writeVolumeUuidFiles = false, or set the environment variable PS_WRITE_VOLUME_UUID_FILES=false.

Photo of Matthew

Hi! I’m Matthew, the author of PhotoStructure. I’m a dad, amateur photographer, and author of open source, adtech, fintech, and edtech software since the turn of the millennium. He/him. I am building a self-hosted, safe, and fast new home for all your photos and videos.
Your memories deserve PhotoStructure. Try it out for free!