--- title: PhotoStructure Tools url: https://photostructure.com/server/tools/ description: Learn about PhotoStructure's tooling to help you work with your library. date: 2020-07-12 --- By installing [PhotoStructure for Node](/server/photostructure-for-node/) or [PhotoStructure for Docker](/server/photostructure-for-docker/) you'll get access to tools that help answer questions about your PhotoStructure library. - Some level of proficiency with the terminal is required to use these tools. - If you have a task that isn't quite covered by any of these examples, feel free to [post a new feature request in our forum](https://forum.photostructure.com/c/feature-requests/7). ## Tools overview and help To get a list of tools available, use `./photostructure --help`. ### For PhotoStructure for Node ```sh cd ~/photostructure-for-servers ./photostructure --help ``` (If you're running [PhotoStructure for Node on Windows](/server/photostructure-for-node/#win10-install), this page assumes you're running these commands within a [Git Bash](https://git-scm.com/download/win) terminal (a `cmd` or PowerShell terminal window won't work). ### For PhotoStructure for Docker ```sh docker exec -u node -it photostructure ./photostructure --help ``` Note that this assumes the docker container's name is "photostructure", and the container is running. You may want to add an alias, say, `alias phstr='docker exec -u node -it photostructure ./photostructure'`, to make this just `phstr --help`. If you don't use [PUID/PGID](/go/puid), omit the `-u node` argument. If you see "Error: The library data directory does not exist" you're probably running `docker run`. You want **`docker exec`**. [See this forum post for details.](https://forum.photostructure.com/t/error-when-using-docker-run-photostructure-server-photostructure-info/520/2) The `su -p node` bit ensures any files that PhotoStructure touches retain correct file permissions. [See this forum post for details](https://forum.photostructure.com/t/files-in-ps-tmp-owned-by-root/1597/2?u=mrm). ### Example help output Here's the output from `./photostructure help` (as of July 2022): ```bib Usage: photostructure [options] [command] Welcome to PhotoStructure, your new home for all your photos and videos. See for details about these tools. Options: -V, --version output the version number -h, --help display help for command Commands: main PhotoStructure's main process manager. Runs and manages web and sync services. (default) info Configuration, file metadata and import diagnostics tool. list List paths in a PhotoStructure Library. logcat Chronologically sort and pretty-print PhotoStructure logfiles. logtail View the log messages of currently-running PhotoStructure processes as they are emitted. (Like `tail -f`). web PhotoStructure's web service. Automatically started by main. sync PhotoStructure's directory synchronization service. Automatically started by main. help [command] display help for command Copyright © 2017-2022, PhotoStructure Inc. BY USING THIS SOFTWARE, YOU ARE ACCEPTING ALL THE TERMS AND CONDITIONS OF THIS LICENSE: User guide: Questions, bug reports, and feature requests: ``` ## Command help To see usage instructions, use `./photostructure help [command]`, like so: ```bib ./photostructure help logcat Usage: logcat [options] [FILE_OR_DIR...] Chronologically sort and pretty-print PhotoStructure logfiles. FILE_OR_DIR may be either a logfile or a directory that will be scanned for .log and .log.gz files. Options: --recent [minutesAgo] Automatically cat recent logs. Defaults to 1 day. --color Enable ASCII terminal colors (default: true) --no-color Disable ASCII terminal colors --version Output the version number (spoiler: it's 1.1.0) -h, --help display help for command ``` ## Logging Know that the default log level is `error`, so expect the logs to be pretty quiet. [See more about log levels and how to adjust them here.](/faq/error-reports/) PhotoStructure encodes logs as JSON, which is a bit hard to read. To pretty-print any logfile, use ```sh ./photostructure logcat $PATH_TO_LOGFILE... ``` ### Recent logs To see recently written logs, use ```bib ./photostructure logcat --recent ``` ### Real time logs To view logs as they are emitted by all currently running PhotoStructure processes, use ```bib ./photostructure logtail ``` ### Paging with `less` You may want to add `| less -R` to the end of either of `logcat` or `logtail` to page and search through the output. The `-R` switch allows `less` to pass ANSI color escape sequences to the terminal. All tools support disabling color via the [NO_COLOR](https://no-color.org/) environment variable or `--no-color` command-line switch. ## Manually importing files and directories You can manually import directories into your PhotoStructure library via the command line. If you are using PhotoStructure for Node, this is easy: ```bib ./photostructure sync [file or directory] ... ``` If you are using PhotoStructure for Docker, you have to "bind mount" the host directory you want imported into your photostructure container, and then run `sync` against that new directory. (This is a lot more convenient if you're using [Docker Compose](/server/docker-compose-wizard/)). ## System information Use the `info` tool without any arguments: ```sh $ docker exec -u node -it photostructure /ps/app/photostructure info ``` 
{
  systemInfo: {
    version: '2.1.0-alpha.3',
    edition: 'PhotoStructure for Servers',
    os: 'Ubuntu 20.04.4 LTS on x64',
    nodeJs: '16.15.0',
    videoSupport: 'FFmpeg 4.2.7-0ubuntu0.1',
    heifSupport: '/usr/bin/heif-convert',
    freeMemory: '30 GB / 67 GB',
    cpus: '24 × AMD Ryzen 9 3900X 12-Core Processor',
    concurrency: 'Target system use: 75% (18 concurrent imports, 4 gfx/process)'
  },
  healthChecks: {
    ok: [ "Library isn't set up yet", 'Memory by info (43 MB) is OK' ],
    warn: [],
    bad: [],
    fail: []
  }
}
## File information Use the `info` tool with a full pathname to a photo or video to see in-depth diagnostic information, including: - When the photo or video was captured - Where the captured-at was extracted or inferred - Dimensions, exposure information, and other extracted metadata - File sorting criteria, used to determine [which file to show](/faq/what-do-you-mean-by-deduplicate/#how-does-photostructure-pick-which-file-to-show) - The filters that "accept" or "reject" the file. If any filters reject the file, it will not be imported into your library. - The validity of a file (if it is `OK`, it is less likely to have suffered from [bit rot](/faq/how-do-i-safely-store-files/#why-3-2-1-isnt-enough-the-impact-of-bit-rot)) An example of a file that's [in a `NoMedia` directory](/faq/how-to-hide-directories/) (note the `rejected: ['noMediaFilter']`): ```bib $ ./photostructure info ~/Photos/should-be-ignored/image.jpg [ { _PhotoStructureVersion: '0.8.2', capturedAt: { localCentiseconds: 2016121309052712, offset: -480, src: 'tags:SubSecDateTimeOriginal' }, dimensions: { width: 4048, height: 3036 }, errors: [], exposureSettings: { focalLength: '4.7 mm', iso: 50, aperture: 2, shutterSpeed: '1/831' }, fileSortCriteria: { resolution: 12, mtime: 13286984, schemeIdx: 2, isCover: false, count: 0, isBrowserSupported: true, uri: 'psfile://mV7DiVCKr/home/mrm/test/image.jpg' }, filters: { accepted: [ 'exifExtFilter', 'fileSizeFilter', 'notHiddenCheap', 'hasMimeType', 'supportedMimeTypeFilter', 'imageHasMakeAndModel', 'notRejected', 'minDimensionsFilter' ], rejected: [ 'noMediaFilter' ] }, geohash: 341153074696371, ignoredBecause: [], imageHash: { meanHash: '9/LwkIQAAgfQchcGav7U8gcHyeH1sSM=', modes: [ 950, 946, 2706, 918, 2707, 2711, 914 ], dominantColors: 'Nickel (#727472), Ebony (#555D50), Grey (#808080), Artichoke (#8F9779), Black olive (#3B3C36)' }, Make: 'Google', mimetype: 'image/jpeg', Model: 'Pixel', nativePath: '/home/mrm/Photos/should-be-ignored/image.jpg', rotation: 0, sha: 'h9SvLXneMYIMttY2dsAoEm820TFpulRa', tags: [ [ 'Camera', 'Google', 'Pixel' ], [ 'When', { name: '2016', ordinal: 7984 }, { name: 'Dec', ordinal: 1 } ], [ 'Type', 'Image', 'JPEG' ] ], tz: 'America/Los_Angeles', tzSource: 'from Lat/Lon', uri: 'psfile://mV7DiVCKr/home/mrm/Photos/should-be-ignored/image.jpg', validFile: 'OK' } ] ``` ## File comparisons Use the `info` tool with **two** full pathnames to photos or videos to determine if PhotoStructure considers those files to be variations of the same asset. Here's an example: ```bib $ ./photostructure info ~/a.jpg ~/b.jpg ... These files differ: low image correlation { af1: { '$ctor': 'models.AssetFile', ... }, af2: { '$ctor': 'models.AssetFile', ... } } ``` ## Path names in your library Although your PhotoStructure library is stored as an [SQLite](https://sqlite.org/index.html) database with an easy-to-read schema design, the cross-platform URIs associated to each asset file have [volume UUIDs](/faq/what-is-a-volume/#volume-uuids-save-the-day) that aren't simple to convert back to a pathname. The `photostructure list` command (added in version 0.8.3) converts file URIs in your database back into full pathnames relevant to the computer you're on. It also lets you filter the output (using SQL) and emit the output either as plaintext or JSON. With `list` and `--where`, you can answer a bunch of questions you may have about your library. The `--json` argument lets you control your output when used in conjunction with [`jq`](https://stedolan.github.io/jq/). The `--dump` argument emits _all_ database fields, encoded in JSON. This is extremely verbose. Note: if you plug in a device, import values, and unplug the device, the path emitted by `list` will be based on the mountpoint when it was visited last. That file won't (currently) exist on your system. ## Show me all the filenames in my library With node: ```sh cd ~/photostructure-for-servers ./photostructure list ``` On docker: ```sh docker exec -u node -it photostructure /ps/app/photostructure list ``` Note these full pathnames include all variants for all assets. If the files are on a currently-unmounted volume, the path emitted will be based on the last mountpoint for that volume, and won't exist on your system. The paths will be emitted in imported order, not alphabetical order. Add `| sort` to your command to alphabetize your output. ## Show me all the "shown" filenames for each asset To emit only [the "shown" or "primary"](/faq/what-do-you-mean-by-deduplicate/#how-does-photostructure-pick-which-file-to-show) filenames for each asset: With node: ```sh cd ~/photostructure-for-servers ./photostructure list --where "Asset.shown=1 AND AssetFile.shown=1" ``` On docker: ```sh docker exec -u node -it photostructure /ps/app/photostructure \ list --where "Asset.shown=1 AND AssetFile.shown=1" ``` ## Show me all the "duplicate" variant filenames for each asset To emit the filenames that are duplicate, non-primary variants: With node: ```sh cd ~/photostructure-for-servers ./photostructure list --where "Asset.shown=1 AND AssetFile.shown=0" ``` On docker: ```sh docker exec -u node -it photostructure /ps/app/photostructure \ list --where "Asset.shown=1 AND AssetFile.shown=0" ``` ## Which files in a directory didn't get imported? This answer takes a couple steps. Let's assume the directory we want to examine is `/opt/Photos`. First use `find` to get the filenames in `/opt/Photos`. Note that in this example we're skipping the `.photostructure` library preview directories, and only select JPEG and Canon raw images (`.CR2`). Please change this command to suit your situation. ```sh find /opt/Photos -iname .photostructure -prune \ -type f -iname \*.jpeg -o -iname \*.jpg -o -iname \*.cr2 \ | sort > ~/all.txt ``` Then find the filenames in your library using `list` and `grep`: ```sh ./photostructure list | grep -E "^/opt/Photos/" | sort > ~/imported.txt ``` Finally, we use [`comm`](https://en.wikipedia.org/wiki/Comm) to show all the files in `all` that aren't in `imported`: ```sh comm -2 -3 ~/all.txt ~/imported.txt ``` For each file that wasn't imported, you can use the [`info`](#file-information) tool to determine why PhotoStructure did not include them in your library (look at the `filters.rejected`, `ignoredBecause`, and `validFile` fields). Also see [why didn't my file get imported into my Library?](/faq/why-is-my-file-missing/) for more information about PhotoStructure's file filters. ## How do I sync folders manually? PhotoStructure runs [`sync`](/server/photostructure-for-servers/#service-architecture) for all directories referenced in your `scanPaths` [system setting](/getting-started/advanced-settings/#system-settings). This setting is (one of the few!) that you can edit via the Settings page. By default, `sync` imports each of these paths serially, syncs your library directory, and then waits for `syncIntervalHours` (a [library setting](/getting-started/advanced-settings/#library-settings)) before restarting. If you want to run sync at a specific time (perhaps, after a backup), run ```sh ./photostructure sync --exitWhenDone ``` (this is for PhotoStructure for Node: for Docker users, replace `./photostructure` with something like `docker exec -u node -it photostructure ./photostructure`). If you want to sync a specific folder (perhaps, just the directory that is backed up), add the directory (or directories) to the command line: ```sh ./photostructure sync /path/to/directory ``` - These directories don't need to be in your `scanPaths` - Run `... sync --help` to describe all command-line arguments. - Any assets found in these directories will be added to your library - If the automatic sync was already running, PhotoStructure will inadvertently "overschedule" processes on your system, although all those processes will be `nice`d. You may want to set the default value of the `cpuLoadPercent` system setting to something like 25 or 50, so periodic scans run slower, and set your manual sync to use a higher value, so your manual scan completes sooner: ```sh PS_CPU_LOAD_PERCENT=75 sync /path/to/directory ``` for example, will only use 75% of your CPUs to ensure your system remains responsive. ## How do I sync a bunch of files manually? The `sync` process accepts filenames, separated by newlines, on `stdin`. If your files have newlines in them (!!), encode the filename in newline-separated JSON objects, like `{"path": "/path/to/filename"}`. The following example force-rebuilds all Assets whose `shown` column is `0`. ```sh ./photostructure list --where "Asset.shown = 0" | \ ./photostructure sync --force --info ``` In case you were wondering: the `Asset.shown` column is used to discriminate between Assets that can be "shown" by the UI (they will have a value of `1`), and those Assets that are _not_ ready to for the UI (because all of the import steps, like transcoding, preview building, and tagging, haven't finished yet).