Your first 100,000 photographs are your worst

At the turn of every year, I try to do a bit of digital housecleaning. It’s nice to do this every once in a while: get all your files, your backups and other security details in order across all your devices and services.

While going through this most recent sweep, I started wondering how best to organize my photos. I’ve taken about 25,000 photos (and only about 900 videos) on my iPhone since 2012. The ones I took from 2007 to 2012 are all in an iPhoto Library file somewhere in a backup drive. So that’s probably another 20,000 photos, conservatively, taken over those years. Then, I easily have another 25,000+ photos in high-resolution form from the various cameras I have owned over the years.

Reviewing this history, I’m reminded of Cartier-Bresson: “Your first 10,000 photographs are your worst.” In the digital age, I think to myself, should this be 100,000 photographs? I think this not only because I don’t believe I’m a very good photographer yet – but also because it might take me another 25,000 photos before we figure out how to safely and effectively store them forever.

For the photos coming from my cameras, I’ve been using Lightroom to organize and store into my backup drive.

Until now, though, the photos taken on iPhone I’ve just been leaving on my phone. However, I started running out of space on the phone. I was backing them up into Dropbox, but I’m out of space there too (really Dropbox, a 1TB limit for personal use? Why?). So I moved them all into Amazon Drive. If you have Prime, you get free unlimited storage of all photo files in hi-res format. It’s definitely the best deal going. The Amazon Photos user interface needs quite a lot of work, but the syncing is so much faster than the other services. (At least, it feels that way to me; maybe it’s just because I have the upload/download bandwidth and concurrent limits set to max.)

It feels like this is a process once every couple of years: pick a service, move everything over to that service, hope that you don’t lose anything, hope that there are no proprietary file formats or file names or strange organization structures. As a part of this review, though, I realized I prefer to organize them in a certain way “on disk”. Because the file sizes are so big, it makes sense not to have them all in one big “Camera Uploads” folder. I group photos into folders by source, and then by year. For instance, each camera gets its own top-level source folder, underneath which photos from each year are grouped. The individual photo files are named according to the date they were taken. This allows me to manually find photos much more quickly, and it allows me to sync only the subset that I want to across devices. A source isn’t just my camera though, because photos friends send me can get their own top-level grouping. I try to make sure whichever editing or backup tool I choose respects this hierarchy. They can hold whatever other metadata they each want, as long as the basic structure is one and the same across applications.

One side effect of this whole reorg is that it basically means that my workflow for my digital camera is now the same as for my iPhone. The phone has truly turned into one of my cameras. As a photographer, no matter how big the memory card, I take all the photos off after a shoot and then I use the best tool possible to organize them in the way I want to organize them and another tool to post-process them the way I want them to look. Why should the latest camera, iPhone X, be any different?

Another side effect is that my Fuji digital camera and the iPhone X are just two cameras, two devices. In fact, the way I see it now is that they are two sets of lenses: just as in the past, I might have carried a wide and a portrait, I’m carrying two lenses now: whatever is on the Fuji and the wide-angle ƒ/1.8 on the iPhone. They both have great lenses, they both have WiFi, they both take great shots I love (and other people do too, I hope) and they both make me creative.

So, in addition to the storage, why shouldn’t the workflow for both be the same? Onto the next 25,000 photos we go.

How do you keep your ever-increasing set of photos organized?

The more you know

A few people sent me the Times article on Strava’s global usage and paths in heat map form this week. Leaving aside the alarming headlines and shares for a second, I wanted to really think through: what are the real issues here? The data on Strava, whether in aggregate form or not, was already publicly available on their site. You can look up certain areas and paths and see all the top people that biked, ran and swam them. I think this feature has been there for as long as the site has existed. The fun of it is that you’ll discover other great athletes near you who run the same patterns – an interesting super-local social network of sorts based on paths. There’s a certain magic there that makes big cities like New York City feel more like a village.

Strava’s contract has always been: join our community, share your paths with other enthusiasts like yourself, maybe learn about new paths near you which you may not have otherwise known. If you wish to opt out and want to keep your data private, Strava’s had a way out. Strava’s settings, like any other social sharing app, make it easy for the user to block their sharing and to keep paths private if you so chose. They are right in offering up those tools and putting the privacy settings back in the hands of the end user, who ultimately should be in control of whether their data is released and shown elsewhere.

The first issue

At the same time however, people don’t know better and are too busy and can’t dig into every setting out there on every platform. There’s a lot of friction involved in getting end users to update settings. Most people just don’t know or don’t care. So if it really matters, the platforms should take care not only to inform them, but to try their best to protect them as well. Many employers, including the government, have to work harder to keep their users informed of the settings and how to protect themselves and their organizations. Can we really trust users with the default settings on any system, knowing they’ll leave the defaults on, knowing they’ll never change the password to their wireless routers?

I’m reminded of a question Apple asks you every so often – which probably makes you think of your battery life more than it does your potential privacy leaks: ‘”________” has been using your location in the background. Do you want to continue allowing this?‘ (Speaking of Apple, take a look at how many location-related settings there are in iOS; you can get very granular with this stuff and still not be able to cover it all.)

The second issue

Why do we trust all our data to Apple & Google fully knowing they can read and hear everything, but then panic over smaller companies and third parties having our information? Is it somehow different if you’re a small service or startup versus a large one?

The third issue

As another exercise: let’s fast-forward and think about a fully decentralized future (I hope), where you are fully in control of your data and you own your data security keys – individually or in aggregate. If a leak happened, would people just blame themselves? People always want someone to blame, but given the choice to manage their own security keys and data, I’m sure a lot of people would not want to deal with it, and would trade for the convenience of data-in-a-centralized-cloud instead.

The fourth issue

Data aggregates can get you into trouble no matter even if the individual data point is harmless. An example: I know where you are right now based on your location in your most recent Instagram or Snapchat story. That’s only one data point, so maybe I can’t do much with it. But if I had a direct API feed of your full history, then that exposes a lot more about your life – patterns and paths and timestamps and locations over time from which I can derive real meaning.

This isn’t new to the world of mobiles and location data either; it is something that comes up in medicine regularly. Here’s a study from PLoS Med on ‘Ethical and Practical Issues Associated with Aggregating Databases‘ from ten years ago:

Participants who consented to the collection of their data for use in a particular study, or inclusion in a particular database, may not have consented to “secondary uses” of those data for unrelated research, or use by other investigators or third parties. There is concern that institutional review boards (IRBs) or similar bodies will not approve of the formation of aggregated databases or will limit the types of studies that can be done with them, even if those studies are believed by others to be appropriate, since there is a lack of consensus about how to deal with re-use of data in this manner.

Combined databases can raise other important ethical concerns that are unrelated to the original consent process. For example, they may make it possible for investigators to identify individuals, families, and groups. Such concerns may be exacerbated in settings where there is the possibility of access to data by individuals who are not part of the original research team.

If you opted out of your individual data, so should you then also have a setting an option to opt out of the “secondary uses” of your data. It’s not enough to let others determine the fate of how they’re anonymizing keys and uniquely identifiable information in datasets. Not only should you be able to opt out of the individual level, but also out of the secondary use process, because you don’t know and don’t have control over that process (and it also likely won’t benefit you or the primary study).

These aren’t easy problems to solve for. They all trade some level of privacy for convenience, for wanting to be noticed, and for wanting free platforms (which are ad-supported, and which then take your data for secondary uses). Ultimately, to properly care for data, it will come down to a new sort of contract between an individual and a service: the individual’s right to hold their data for their own use, the individual’s right to take back their data when they so choose and their right to be forgotten altogether.

The endgame (and AlphaGo)

I recently watched the movie AlphaGo on Netflix, which documents the lead up to and the challenge match between DeepMind’s AlphaGo and Lee Sedol.

Three things jumped out at me watching the movie.

One, we know it’s going to happen. Even if you didn’t previously know the outcome of AlphaGo versus Sedol, you know someday soon, the computer is going to surpass and beat the human. If not this version, then the next. If it’s only able to win a few games now, it will win all games in future. We already know the ending. But you just can’t help but want Sedol to win, because it means that we all win. It means that we put off by another day the inevitable moment when the computer can beat us. Not just beat us by being faster than our brains and bodies, like previous inventions, but by learning by itself and out-thinking us.

Two, the way the documentary protrays the tension between the two sides is when it strikes you that nobody thought it would happen so soon. Neither side, DeepMind or Sedol, thought so even as it was unfolding. It’s a moment that is simultaneously terrifying and heartbreaking and amazing at the same time. The endgame is near, but how amazing that we were able to get a group of people together to program that.

Three, and this is on your mind as you watch and this is why the movie is so good: it asks the question “What happens when it happens?” What’s it do the psychology of humans to know they can’t win and that the computer has surpassed them? What’s the emotional toll? Sedol, even with his incredible winning percentage has lost other games to human players in the past, but this loss is just so much different: knowing he (and, therefore, all us humans) can’t win this one. And what happens after it happens? Will humans just play human games and computers computer games?

In the end, we’re left with only a hope. A hope that the creativity of the machine will unlock a new creativity within us. To allow us to see moves and the world in new ways we hadn’t before envisioned. And like any other tool we’ve invented – the pencil, the bicycle, the car – the computer will continue doing just that.

A thousand true fans

I came across two great remembrance posts this afternoon, from Om and from Kottke, about the loss of their friend Dean Allen.

I never met him but I was always a fan of his work, from textism to Textpattern to favrd. When he first announced TextDrive back in 2004, I put up the $200 (a lot for that time in life) and signed up to support him and get a lifetime account out of it. We talk a lot these days about crowdfunding and bootstrapping your work via your first thousand true fans – TextDrive was the first time I’d ever encountered that concept on the web. It was a refreshing and beautiful thing.

I’m always amazed that one person can leave so much behind on the internet – and to keep trying new things, just for the sake of trying things, and to move the web forward because that’s the only way it ever does.

It seems a thousand others thought so today as well.


Earlier this week, at a team dinner, we got to talking about whether people can and will ever move off of Facebook. The question was posed not necessarily as an exercise into how to start a new social network, but to talk about the company’s influence and network power, and to ask: if something else took its place, wouldn’t that service naturally go through the same evolution? Create a hook, aggregate people and data and attention, and then monetize that attention with advertising. The same question can be posed of all institutions that grow too big and centralize power: aggregate a resource, attract people in, have network effects or other data lock-in to make it hard for them to leave.

When it comes to big companies, you can only believe in one of three things: 1) that if something grew too big, people will always have the freedom to vote with their feet and move on; 2) that government will step in and break it up; 3) that technology will improve to break us out of such lock-ins because technology wants to innovate and improve itself (it could be argued that the default state of technology is to always be in a process of self-improvement like this).

On the first, that we are free to choose whichever service, we know now is increasingly not true: with almost all types of centralization, and especially with network effects businesses, you can’t just move on easily. You’re on it because your friends are on it, and they’re on it because you’re on it. You’re on it, because it lists more products than the other sites and it already has your billing information, and that makes it more convenient. And because more people are shopping on it, more products and vendors get on it. So, the more powerful each network becomes, the harder it is to remove yourself (and your data) from it. The “well, if they increase prices, I’ll just go elsewhere” is just not true these days, because at some point a service gets so big that there aren’t many good alternatives that will have the stuff you need.

You can believe that governments will step in and do what they did in previous eras to fight anti-trust and break up things that get too big. You can believe that policy will change, but it may not always go your way (e.g., the state of the net neutrality debate now). Policy is always reflective of who’s running the government during any span of years. It can go one direction for a while, and then, as we see now, flip into another for a few years. So, you can’t solely believe in and wait for that either. (In fact, even when it does feel like it worked, as with Microsoft in the 90s, it could be argued it wasn’t just government intervention that made them stumble, but rather that they missed multiple technology trends in a row).

So ultimately, it leaves it up to technology to provide solutions for us and that’s why for those that are working in and watching the space, decentralized technologies like cryptocurrencies are so interesting.

In New York Times Magazine this weekend, Steven Johnson has a beautifully written essay on bitcoin, blockchain, the underlying technologies behind networks and why there could be a ‘there’ there. Like a great photo or idea, it’s one of those things you wish you had done, that you had written it this way for others to understand – that was my first reaction! He explains it better than any of hundreds of other posts and articles you’ll read on this stuff. Now when anyone asks me for a starter into networks and cryptocurrencies, I will just send them this essay as the first read.

You have to believe in one of these three things. Each has people behind the scenes, creating policies and movements and supporting each style of action. But it’s the belief in technology that has consistently moved us forward in the past, and the belief is that it will once more, and that it should.

Cool tools

I’m always on the hunt to find cool apps to add to my collection. Part of it is driven by the need to find faster or more efficient tools. Part of it is the desire to find more beautifully designed experiences and maybe learn about how developers are putting these things together.

Here are a few apps I’ve found recently that have been a great help in productivity on the desktop as well as my iPhone. Bonus: I realized it after I typed these all up, but they are all free!

  • Numi – A beautiful, minimal app for macOS that makes you feel as though this is the way the calculator should have worked all along. It seamlessly blends math with text. It reminds me of those ticker tape calculators from years past, except this one has syntax highlighting, understands units to make conversion easier (forex, crypto, time, data size), variables so you can reuse calculations, percentages and simple math functions (like factorial, round).
  • Spectacle – A fast, simple window manager for organizing windows on your Mac. After having tried a few of these over the years, what I like is that the app is really lightweight and the keybindings are really intuitive and “just work” out of the box. This way, you don’t really have to customize every single view and can install it on every machine you have and have them all behave the same.
  • Chime – I’ve long been looking for a simple watch (or watch app) that simply vibrates every few {five, fifteen, thirty, or sixty} minutes on my wrist. I found this simple app for iOS that does this well, all while doing so with a simple interface and no stray notifications in your drawer. I tend to keep all sounds off on my phone (texts, apps, etc), but I’ve long wanted to have little reminders throughout the day to help let me how quickly time is passing. As my wife often likes to say, “All we have is time.
  • exiftool – A command-line app that reads, writes, creates files and metadata in a huge number of file formats. It support all types of images, RAWs from cameras, PDFs, etc. It’s great when you want to see what metadata is attached to a particular file; even better when you want to strip some of the properties before posting or sending the file along to someone.

You can find all of these on kit too:

MLK Day and Books Through Bars

“We may have all come on different ships, but we’re in the same boat now.” – Martin Luther King, Jr.

We didn’t get a chance to take the day off today, so tonight we took a little time to give back. Team Kit and I volunteered to help NYC Books Through Bars at Freebird Books. Books Through Bars is an all-volunteer-run group that sends free, donated books to incarcerated people. They mail book packages to individuals who write in with different requests rather than send them to prison libraries. They’ve been doing this for twenty-one years and ship to forty states (the other ten don’t allow such packages through the mail, especially I think from people you don’t previously know). Tonight, we mostly mailed individuals in Texas, a state that also comes with its own list of banned books. This list was hundreds of titles long, and you have to cross-check the requests before you pack them up. Wherever I could find one on the shelf, I threw in one or two of my favorite books as well. We chose to work with this group tonight because we love reading so much ourselves, and we believe in their vision that books are a way to improve quality of life, and access to knowledge shouldn’t be restricted to anyone, by anyone.

The Burn

At the end of every year, Diana and her closest friends get together for “The Burn”. The Burn starts by writing down all the things that went right and all the things that went wrong the past year, all the things you said you were going to do, but you didn’t and all the things you said you were going to do, and you did. You also write down the goals you hope to achieve in the year coming up. Then you sit and talk through the lists with one another, preferably on a beautiful beach somewhere, and you light them all on fire, preferably before beach patrol runs over to wave you down.

This year’s Burn was on Santa Monica Beach.

Santa Monica

An ice-cold weekend in New York City walking the pup has got me thinking about a beach walk in Santa Monica a couple of weeks back.