Uncertain, lazy, forgetful, & impatient: It’s what you want your code to be.

While building PhotoStructure, which is written in TypeScript, I found myself missing a bunch of Scala-isms.

Character trait #1: Unabashed uncertainty 🔗

One of the first bits I missed from Scala was the Option monad.

“Oh no,” I hear you thinking,

but it’s not going to be one of those blog posts. Honest.

For the uninitiated, null pointers have been considered a “billion dollar mistake.”

… I couldn’t resist the temptation to put in a null reference, simply because it was so easy to implement. This has led to innumerable errors, vulnerabilities, and system crashes, which have probably caused a billion dollars of pain and damage in the last forty years.

These are commonly seen as TypeErrors in JavaScript. You’ve probably got a couple in your console right now. Well, maybe not now now, I mean, I hope not.

Functional programming, or FP, has had a solution to this, called Option. An Option is a union type of either

1. Some value, or a
2. None, which the absence of a value.

The important takeaway: when you return an Option (or a Maybe), you’re advertising to your callers that you’re uncertain you can return a value, and that they need to behave accordingly.

Every method that makes an RPC call, handles user data, or in any way touches external systems should probably not be certain that they can be successful in returning something reasonable.

In Ye Olde Java Shoppe, you’d probably have your method throw an Exception. You then have to decide (and fight with your code reviewers) if you need a new Exception subclass, and if it needs to be a run-time or caught exception. If it warrants a new error type, do you subclass the prior typed Exception or introduce another hierarchy? It’s a royal PITA, the fact that it’s arguable means there isn’t really a “right” answer, and, most importantly, your callers probably don’t care why your failed, just that you did. If your caller doesn’t care why you failed (because it doesn’t need to tell their caller), an Option is a nice … option.

Here is a simple implementation of Option in TypeScript. There’s an entire library, fp-ts, which has a rich set of FP tools for you to use, too.

If you’re concerned about overwhelming your garbage collector with these Some wrappers, you can use a much lighterweight approach. Meet Maybe:

export type Maybe<T> = T | undefined;
export type MaybeNull<T> = Maybe<T> | null;

To help with the ergonomics of dealing with Maybes, map is convenient:

export function map<T, U>(obj: MaybeNull<T>, f: (t: T) => U): Maybe<U> {
  return obj == null ? undefined : f(obj);
}

In use:

return map(await db.fetchUser(username), user => {
  // do something interesting with user,
  // now that you know it’s defined
  ...
});

Thanks to the new optional chaining syntax in TypeScript 3.7, if you just want to call a method on the user, you can do something like

(await db.fetchUser(username))?.methodOnUser();

If you want to do more than one thing with the optional reference, the map is still a handy tool to have.

Uncertainty of the future 🔗

But, you say, but everything these days are Promises, what have you got for me to help with that?

Meet thenOpt:

return thenOpt(db.fetchUser(username))
  .flatMap(user => checkForPwnedPassword(user))
  .flatMap(pwnedResult => ...)
  .getOrElse(() => {
    // this block is called if the resolved db user was undefined/null,
    // or the checkForPwnedPassword promise resolved to undefined/null.
  })

It’s the same interface as an Option, but the functions may return synchronous results or Promises. If the Promise resolves to nullish, the remaining calls (until an orElse or getOrElse) are not invoked.

Here is the implementation of thenOpt. Share and enjoy.

Character trait #2: Only paying when it’s due 🔗

Another tool I missed from Scala was lazy, which is a val sub type.

A lazy val is only computed and assigned at access. If that computation is expensive, it’s nice to be able to defer that computation until it’s actually needed. This can both help expedite system startup time as well as work around circular dependencies (when they can’t be, or just aren’t, avoided).

My first implementation was simple, and simply caches a thunk:

export function lazy<T>(thunk: () => T): () => T {
  let invoked = false;
  let result: T;
  return () => {
    if (!invoked) {
      invoked = true;
      try {
        result = thunk();
      } catch (_) {}
    }
    return result;
  };
}

Note that this implementation swallows errors. You may find it more useful to both cache the result, and catch and rethrow any caught error.

A couple use cases:

class File {
  ...
  readonly sha = lazy(() => {
    // Only compute the SHA if someone asks:
    // (implementation left as an exercise for the reader)
  })
}

class Server {
  ...

  readonly end = lazy(() => {
    // cleanup code that must only be called once per instance
  })
}

I found fairly quickly, though, that tests needed to clear prior test state, and that certain lazy fields should probably expire after a certain amount of time, and gee whiz wouldn’t it be nice if I could clean up any resources from prior value but only if there was a prior value, and… Anyway, that fancier lazy is here.

We now say goodbye to the “I miss Scala” portion of our broadcast.

Character trait #3: What are we talking about again? Oh, right. Forgetfulness. 🔗

Many morsels of computation that we want to cache are only valuable or relevant for a short amount of time. If we could allow our program to be appropriately forgetful, we can limit our memory consumption and prevent leaks.

It’s not always easy, though.

There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors. — Leon Bambrick

If you know what you’re caching, and have sufficient domain knowledge to know when it’s appropriate to be forgetful, this problem is tractable.

One simple approach to invalidation is “time-to-live”, or TTL-based invalidation. Caching based on keys is supported by TTLMap.

As a concrete example, if you’re processing files,say, photos and videos, and you know you’ll be done processing the file in probably seconds, but maybe a minute, you can set your TTL on your file metadata cache to a minute, and get excellent cache hit rates.

There are a number of other cache invalidation strategies, but those can wait for another blog post.

Character trait #4: Impatience 🔗

As soon as your method or system requires external resources, as discussed above, the success of your code is not inevitable. Networks drop packets. Filesystems hang. Operating systems decide they don’t need to keep operating. Bad stuff happens.

Sometimes it’s reasonable to pass those hardships onto your callers, but you aren’t like that are you? You’re an upstanding, empathetic designer of ergonomic and user friendly systems. What to do?

Timeouts and retries are a great way to add robustness to otherwise flaky systems, but, as with all things, there are gotchas.

Unfortunately in esNext land you can’t “time out” your promise and release resources, but you can let the caller receive a rejection if the promise is not resolved within a given amount of time.

export async function thenOrTimeout<T>(p: Promise<T>, timeoutMs: number) {
  let resolved = false;
  let result: Maybe<T>;
  p.then((ea) => {
    resolved = true;
    result = ea;
  });
  return Promise.race([
    p,
    new Promise((resolve, reject) =>
      setTimeout(
        () => (resolved ? resolve(result) : reject("timeout")),
        timeoutMs
      )
    ),
  ]);
}

Add logging and season to taste. But yeah. Add timeouts to everything.

You can also add retries pretty easily, too, just be careful to only retry idempotent actions. Here’s an implementation for retryOnReject.

• coding

Hi! I’m Matthew, the author of PhotoStructure. I’m a dad, amateur photographer, and author of open source, adtech, fintech, and edtech software since the turn of the millennium. He/him. I am building a self-hosted, safe, and fast new home for all your photos and videos.
Your memories deserve PhotoStructure. Try it out for free!