PhotoStructure tools

By installing PhotoStructure for Node or PhotoStructure for Docker you’ll get access to tools that help answer questions about your PhotoStructure library.

• Some level of proficiency with the terminal is required to use these tools.

• If you have a task that isn’t quite covered by any of these examples, feel free to post a new feature request in our forum.

Tools overview and help

To get a list of tools available, use ./photostructure --help.

For PhotoStructure for Node

cd ~/photostructure-for-servers
./photostructure --help

(If you’re running PhotoStructure for Node on Windows, this page assumes you’re running these commands within a Git Bash terminal (a cmd or PowerShell terminal window won’t work).

For PhotoStructure for Docker

docker exec -u node -it photostructure ./photostructure --help

Note that this assumes the docker container’s name is “photostructure”, and the container is running.

You may want to add an alias, say, alias phstr='docker exec -u node -it photostructure ./photostructure', to make this just phstr --help.

If you don’t use PUID/PGID, omit the -u node argument.

If you see “Error: The library data directory does not exist” you’re probably running docker run. You want docker exec. See this forum post for details.

The su -p node bit ensures any files that PhotoStructure touches retain correct file permissions. See this forum post for details.

Example help output

Here’s the output from ./photostructure help (as of July 2022):

Usage: photostructure [options] [command]

Welcome to PhotoStructure, your new home for all your photos and videos.
See </server/tools/> for details about these tools.

Options:
  -V, --version   output the version number
  -h, --help      display help for command

Commands:
  main            PhotoStructure's main process manager. Runs and manages web
                  and sync services. (default)
  info            Configuration, file metadata and import diagnostics tool.
  list            List paths in a PhotoStructure Library.
  logcat          Chronologically sort and pretty-print PhotoStructure
                  logfiles.
  logtail         View the log messages of currently-running PhotoStructure
                  processes as they are emitted. (Like `tail -f`).
  web             PhotoStructure's web service. Automatically started by main.
  sync            PhotoStructure's directory synchronization service.
                  Automatically started by main.
  help [command]  display help for command

Copyright © 2017-2022, PhotoStructure Inc.

BY USING THIS SOFTWARE, YOU ARE ACCEPTING ALL THE TERMS AND CONDITIONS OF THIS
LICENSE: </eula/>

User guide: </user-guide/>

Questions, bug reports, and feature requests:
<https://forum.photostructure.com/>

Command help

To see usage instructions, use ./photostructure help [command], like so:

./photostructure help logcat

Usage: logcat [options] [FILE_OR_DIR...]

Chronologically sort and pretty-print PhotoStructure logfiles.

FILE_OR_DIR may be either a logfile or a directory that will be scanned for .log
and .log.gz files.

Options:
  --recent [minutesAgo]  Automatically cat recent logs. Defaults to 1 day.
  --color                Enable ASCII terminal colors (default: true)
  --no-color             Disable ASCII terminal colors
  --version              Output the version number (spoiler: it's 1.1.0)
  -h, --help             display help for command

Logging

Know that the default log level is error, so expect the logs to be pretty quiet. See more about log levels and how to adjust them here.

PhotoStructure encodes logs as JSON, which is a bit hard to read. To pretty-print any logfile, use

./photostructure logcat $PATH_TO_LOGFILE...

Recent logs

To see recently written logs, use

./photostructure logcat --recent

Real time logs

To view logs as they are emitted by all currently running PhotoStructure processes, use

./photostructure logtail

Paging with less

You may want to add | less -R to the end of either of logcat or logtail to page and search through the output. The -R switch allows less to pass ANSI color escape sequences to the terminal.

All tools support disabling color via the NO_COLOR environment variable or --no-color command-line switch.

Manually importing files and directories

You can manually import directories into your PhotoStructure library via the command line.

If you are using PhotoStructure for Node, this is easy:

./photostructure sync [file or directory] ...

If you are using PhotoStructure for Docker, you have to “bind mount” the host directory you want imported into your photostructure container, and then run sync against that new directory. (This is a lot more convenient if you’re using Docker Compose).

System information

Use the info tool without any arguments:

$ docker exec -u node -it photostructure /ps/app/photostructure info

{
  systemInfo: {
    version: '2.1.0-alpha.3',
    edition: 'PhotoStructure for Servers',
    os: 'Ubuntu 20.04.4 LTS on x64',
    nodeJs: '16.15.0',
    videoSupport: 'FFmpeg 4.2.7-0ubuntu0.1',
    heifSupport: '/usr/bin/heif-convert',
    freeMemory: '30 GB / 67 GB',
    cpus: '24 × AMD Ryzen 9 3900X 12-Core Processor',
    concurrency: 'Target system use: 75% (18 concurrent imports, 4 gfx/process)'
  },
  healthChecks: {
    ok: [ "Library isn't set up yet", 'Memory by info (43 MB) is OK' ],
    warn: [],
    bad: [],
    fail: []
  }
}

File information

Use the info tool with a full pathname to a photo or video to see in-depth diagnostic information, including:

• When the photo or video was captured
• Where the captured-at was extracted or inferred
• Dimensions, exposure information, and other extracted metadata
• File sorting criteria, used to determine which file to show
• The filters that “accept” or “reject” the file. If any filters reject the file, it will not be imported into your library.
• The validity of a file (if it is OK, it is less likely to have suffered from bit rot)

An example of a file that’s in a NoMedia directory (note the rejected: ['noMediaFilter']):

$ ./photostructure info ~/Photos/should-be-ignored/image.jpg
[
  {
    _PhotoStructureVersion: '0.8.2',
    capturedAt: {
      localCentiseconds: 2016121309052712,
      offset: -480,
      src: 'tags:SubSecDateTimeOriginal'
    },
    dimensions: { width: 4048, height: 3036 },
    errors: [],
    exposureSettings: {
      focalLength: '4.7 mm',
      iso: 50,
      aperture: 2,
      shutterSpeed: '1/831'
    },
    fileSortCriteria: {
      resolution: 12,
      mtime: 13286984,
      schemeIdx: 2,
      isCover: false,
      count: 0,
      isBrowserSupported: true,
      uri: 'psfile://mV7DiVCKr/home/mrm/test/image.jpg'
    },
    filters: {
      accepted: [
        'exifExtFilter',
        'fileSizeFilter',
        'notHiddenCheap',
        'hasMimeType',
        'supportedMimeTypeFilter',
        'imageHasMakeAndModel',
        'notRejected',
        'minDimensionsFilter'
      ],
      rejected: [ 'noMediaFilter' ]
    },
    geohash: 341153074696371,
    ignoredBecause: [],
    imageHash: {
      meanHash: '9/LwkIQAAgfQchcGav7U8gcHyeH1sSM=',
      modes: [
        950,  946, 2706,
        918, 2707, 2711,
        914
      ],
      dominantColors: 'Nickel (#727472), Ebony (#555D50), Grey (#808080), Artichoke (#8F9779), Black olive (#3B3C36)'
    },
    Make: 'Google',
    mimetype: 'image/jpeg',
    Model: 'Pixel',
    nativePath: '/home/mrm/Photos/should-be-ignored/image.jpg',
    rotation: 0,
    sha: 'h9SvLXneMYIMttY2dsAoEm820TFpulRa',
    tags: [
      [ 'Camera', 'Google', 'Pixel' ],
      [
        'When',
        { name: '2016', ordinal: 7984 },
        { name: 'Dec', ordinal: 1 }
      ],
      [ 'Type', 'Image', 'JPEG' ]
    ],
    tz: 'America/Los_Angeles',
    tzSource: 'from Lat/Lon',
    uri: 'psfile://mV7DiVCKr/home/mrm/Photos/should-be-ignored/image.jpg',
    validFile: 'OK'
  }
]

File comparisons

Use the info tool with two full pathnames to photos or videos to determine if PhotoStructure considers those files to be variations of the same asset.

Here’s an example:

$ ./photostructure info ~/a.jpg ~/b.jpg

...

These files differ: low image correlation {
  af1: {
    '$ctor': 'models.AssetFile',
...
  },
  af2: {
    '$ctor': 'models.AssetFile',
...
  }
}

Path names in your library

Although your PhotoStructure library is stored as an SQLite database with an easy-to-read schema design, the cross-platform URIs associated to each asset file have volume UUIDs that aren’t simple to convert back to a pathname.

The photostructure list command (added in version 0.8.3) converts file URIs in your database back into full pathnames relevant to the computer you’re on. It also lets you filter the output (using SQL) and emit the output either as plaintext or JSON.

With list and --where, you can answer a bunch of questions you may have about your library.

The --json argument lets you control your output when used in conjunction with jq.

The --dump argument emits all database fields, encoded in JSON. This is extremely verbose.

Note: if you plug in a device, import values, and unplug the device, the path emitted by list will be based on the mountpoint when it was visited last. That file won’t (currently) exist on your system.

Show me all the filenames in my library

With node:

cd ~/photostructure-for-servers
./photostructure list

On docker:

docker exec -u node -it photostructure /ps/app/photostructure list

Note these full pathnames include all variants for all assets. If the files are on a currently-unmounted volume, the path emitted will be based on the last mountpoint for that volume, and won’t exist on your system.

The paths will be emitted in imported order, not alphabetical order. Add | sort to your command to alphabetize your output.

Show me all the “shown” filenames for each asset

To emit only the “shown” or “primary” filenames for each asset:

With node:

cd ~/photostructure-for-servers
./photostructure list --where "Asset.shown=1 AND AssetFile.shown=1"

On docker:

docker exec -u node -it photostructure /ps/app/photostructure \
  list --where "Asset.shown=1 AND AssetFile.shown=1"

Show me all the “duplicate” variant filenames for each asset

To emit the filenames that are duplicate, non-primary variants:

With node:

cd ~/photostructure-for-servers
./photostructure list --where "Asset.shown=1 AND AssetFile.shown=0"

On docker:

docker exec -u node -it photostructure /ps/app/photostructure \
  list --where "Asset.shown=1 AND AssetFile.shown=0"

Which files in a directory didn’t get imported?

This answer takes a couple steps. Let’s assume the directory we want to examine is /opt/Photos.

First use find to get the filenames in /opt/Photos. Note that in this example we’re skipping the .photostructure library preview directories, and only select JPEG and Canon raw images (.CR2). Please change this command to suit your situation.

find /opt/Photos -iname .photostructure -prune \
  -type f -iname \*.jpeg -o -iname \*.jpg -o -iname \*.cr2 \
  | sort > ~/all.txt

Then find the filenames in your library using list and grep:

./photostructure list | grep -E "^/opt/Photos/" | sort > ~/imported.txt

Finally, we use comm to show all the files in all that aren’t in imported:

comm -2 -3 ~/all.txt ~/imported.txt

For each file that wasn’t imported, you can use the info tool to determine why PhotoStructure did not include them in your library (look at the filters.rejected, ignoredBecause, and validFile fields).

Also see why didn’t my file get imported into my Library? for more information about PhotoStructure’s file filters.

How do I sync folders manually?

PhotoStructure runs sync for all directories referenced in your scanPaths system setting. This setting is (one of the few!) that you can edit via the Settings page.

By default, sync imports each of these paths serially, syncs your library directory, and then waits for syncIntervalHours (a library setting) before restarting.

If you want to run sync at a specific time (perhaps, after a backup), run

./photostructure sync --exitWhenDone

(this is for PhotoStructure for Node: for Docker users, replace ./photostructure with something like docker exec -u node -it photostructure ./photostructure).

If you want to sync a specific folder (perhaps, just the directory that is backed up), add the directory (or directories) to the command line:

./photostructure sync /path/to/directory

• These directories don’t need to be in your scanPaths
• Run ... sync --help to describe all command-line arguments.
• Any assets found in these directories will be added to your library
• If the automatic sync was already running, PhotoStructure will inadvertently “overschedule” processes on your system, although all those processes will be niced. You may want to set the default value of the cpuLoadPercent system setting to something like 25 or 50, so periodic scans run slower, and set your manual sync to use a higher value, so your manual scan completes sooner:

PS_CPU_LOAD_PERCENT=75 sync /path/to/directory

for example, will only use 75% of your CPUs to ensure your system remains responsive.

How do I sync a bunch of files manually?

The sync process accepts filenames, separated by newlines, on stdin. If your files have newlines in them (!!), encode the filename in newline-separated JSON objects, like {"path": "/path/to/filename"}.

The following example force-rebuilds all Assets whose shown column is 0.

./photostructure list --where "Asset.shown = 0" | \
  ./photostructure sync --force --info

In case you were wondering: the Asset.shown column is used to discriminate between Assets that can be “shown” by the UI (they will have a value of 1), and those Assets that are not ready to for the UI (because all of the import steps, like transcoding, preview building, and tagging, haven’t finished yet).

• PhotoStructure for Servers
• Getting Started