Whenever I move to a new environment, there’s always a period of learning the quirks and unstated assumptions of that new environment. This learning period exists whether the new environment is “graduate school”, or “this web application being built by this team.” During this learning period, you can’t expect to get much done.
In Cincinnati, Ohio, people will say “Please?” when they really mean “I didn’t understand what you said, could you repeat that?”
When you are working on a project made with gnu autotools, you have to type “libtoolize; aclocal; autoconf; automake — add-missing; ./configure; make;” when you really mean “./configure; make”
Software is the world in microcosm. Almost all cultures have explicit documented knowledge, but the biggest barriers to entry are the soft skills that frustrate your ability to get anything done at first. Fully learning “Do not make eye contact, use a light grip on your handshake” is the same kind of experiential process as “address arrays from 1 instead of 0”. You’ll make a bunch of mistakes at first, and these mistakes will make it harder for you to accomplish your goals.
Doing anything by your self is much easier than getting a group of people to work together towards that goal. Doing anything on a single processor is much easier than doing it over multiple processors. In both cases, the added difficulties arise from communication, and weird error cases.
When you do it by yourself, you know how you want it done, and you don’t have to spend time or energy communicating with yourself. You know your own quirks — your own “internal culture” — and so you don’t do anything that upsets yourself.
Likewise, when you write code running on a single machine, you can make a number of assumptions that are reasonable — function calls will either successfully complete, or throw an exception. Memory will be accessed in a reasonable amount of time, and in the worst case scenario, you can just stop the program and start it again.
When you work with a group, a lot of energy and time is spent coordinating the group; making sure each member of the group understands the portion of the task they are meant to carry out — as well as how they are to interact with other people working on the same task.
If you are having an ‘off day’ but everybody else needs you, you can’t just phone it in. Even when you’re sick, you have to be able to respond to emails and calls in case someone else is stuck. If one of the nodes in your system is buggy, and you turn it off — you might cause a pileup in other parts of the system, which will expose more bugs and break more things.
Netflix is famous for having a part of their system that randomly turns off other programs. The purpose of that system — called “Chaos Monkey” — is to ensure that they function correctly even when failures occur, because failures are inevitable. Is it any surprise that they also offer unlimited parental leave? People getting pregnant or sick or losing time can’t slow the company down too much if that’s part of the design pattern. A hard drive failing is only a problem if you expected no hard drives to fail.
Most companies have the notion of “people who can’t get hit by a bus, because if they did, the whole company would shut down.” This is what we call a single point of failure in distributed systems.
JIRA — a task management system used by programmers — is not too different from a task scheduler in a distributed system. They both have the same end goal — get the tasks done — and they both use the same method — parallelizing independent tasks — to do it.
If you’ve ever worked on a project with a team that’s halfway across the globe, you’ll understand the frustration programmers have when dealing with clock drift in distributed systems.
Software is the world in microcosm.
Starting a company used to take a bunch of money to get off the ground and get going. You’d do a bunch of planning, get your product ready, and get it to market with a big ad push — and either sink or swim at that point.
In software, we call that the waterfall method.
At the same time we see companies adopting agile methodologies to make small changes in software, iterating and making tiny improvements — we see entire companies using the same approach. They put out an ‘MVP’, see how it works, and iterate from there.
In human systems, we call those ‘startups’.
A new company starting out back then had to do almost all the work themselves — now we can use a plethora tiny services offered by different companies, to solve specialized problems without doing that work ourselves. Back in the day, programmers would write their own memory allocators, parsers and other libraries — now we can call on a plethora of open source offerings.
Human systems can outsource HR to zenefits, marketing to Facebook and Google, communication to slack and twilio. Software can use bootstrap for a basic visual look on your website, django for a web stack, and open source libraries for just about everything — except for what your actual contribution is.
Software will enable each of us to contribute what we do best to the world — everything else is just details. In the past, executing on the details was everything. Now we have computers to do that for us.
Our processors used to consume way more power than they do now — just like our machines for manufacturing and transportation. A more power efficient microprocessor and a more efficient engine both have the same end goal — but one took a lot longer to develop than the other.
Software is the world in microcosm — it’s the world reflecting itself, like the internal representations in my mind reflect the external world and my place in it. As Douglass Hofstadter says, the perceptual loop created by representations of the external world beginning to represent themselves is the basis of a self.
Software has built that self for our world, and is now learning to move its fingers and toes through Drones and Uber, motion as a service. The world has a mind, and it runs on TCP. The reliable thoughts you have, expressed in words — those are like TCP. Your underlying emotional communication is like the lower IP layer.
Software eating the world is the snake eating its own tail.
If you’re interested in this kind of thinking, I gave a talk about this concept which explores these connections further.