False Assumptions Considered Harmful
The goal of this essay is to convince you that if you want secure software, you should work for social justice. I realize that this claim may seem outlandish. The argument is simple:
- Buggy software is insecure software.
Exploit pathways are bugs. When a program behaves in a way that its authors did not anticipate, that unanticipated behavior can be exploited by hackers. Software which has no bugs cannot be hacked.
- Writing bug-free software requires a relentless search for false assumptions, and a constant practice of empathy.
While you are writing software, if you make a false assumption about how your system works, that false assumption will give rise to small, hard to find bugs. The only way to write bug-free software is to move slowly, to inspect each of your thoughts for accuracy, and to ferret out things you are implicitly believing without being aware of it.When you work on distributed systems, you must understand how all parts of those systems function well enough to anticipate the way they will respond to your code. This requires the constant practice of simulating other machines in your thoughts, and imagining how these machines will respond to your code. When you do this with people – imagining how they will respond to your actions – we call that empathy. The repeated practice of empathizing with other machines and architectures bleeds over into your every day life. When I started working on distributed systems, I started seeing failure modes in human interaction directly analogous to the failure modes in large software systems.
Perhaps this makes me odd, but I learned to empathize by imagining other people as being machines running a large set of protocols, some of which I shared and some I did not. Port scanning and buffer overflows work just as well on people as they do on metal hardware.
- A relentless search for false assumptions and constant practice of empathy are the basis of Justice.
All societies claim to be just. An unjust society must invent a myth for itself, to explain why its disenfranchised members are being treated fairly. The truth, then, is a threat to unjust societies because it exposes the inconsistencies in the myths that these unjust societies use to justify the oppression of their marginalized citizens.People who consistently search themselves for false assumptions, and reflexively ask “how will this other system interpret my words?” do not restrict this behavior to the coding world. The experience of being repeatedly and emphatically wrong is part of learning mathematics – and it trains you to be skeptical of anyone who is confident without the evidence of back up their beliefs.
The Truth Shall Set Us Free
Alan Turing could not have done what he did in the Nazi regime. The allies won the war because we had better crypto, and we had better crypto because we had better tolerance for weirdos. We had better tolerance because we were more a just society. That is not an accident – it appears to be a fundamental property of existence. A king who feeds his people will have a better army than a king whose people are starving. The army was the first to integrate, because when it’s kill or be killed, anything but the truth is a disaster. Only peacetime affords us the painful luxury of ignorance.
Unstated but firmly held assumptions cause people to do hurtful things to each other, and they cause people to write buggy software. You can’t have a world full of people treating each other like crap without being aware of it, who then somehow instantly attain the awareness necessary to avoid race conditions while querying distributed databases. Ignorance doesn’t partition itself for you.
“Everyone knows when I say that an idea sucks, I don’t mean it personally. Anyone can feel comfortable telling me that I’m being too aggressive. I always listen when people tell me I may be wrong” – those are the same kinds of errors in thinking as “The database will always finish processing this portion of the query before the next one starts, the network failure will always be trapped at this level of the code, this lock will always keep other threads from writing to my data structure and breaking the invariants.”
If you are not free to criticize the leader, you cannot point out that his request is impossible – and so the leader will waste limited resources on an impossible task. If you cannot accept that a person you’ve offended is not acting maliciously, and honestly feels hurts by what you said, you’ll see a nonexistent virus trashing your database, when it’s really just your own code.
I want to live in a just world. I want to live in a fair world. I have more hope that justice will come about through the profit mechanism than social action – and I can imagine a lot of executives are sitting around wondering whether they’re going to be hacked next. So if you care about software security, check yourself for assumptions you have about the human condition that are false. Work to assume someone who feels offended is legitimately hurt by your words, rather than taking the easy way out and telling yourself they are malicious. Before you speak, ask yourself how someone else will respond to your words – asking how you’d respond is not enough, any more than asking how your favorite database will interpret your query, when you are working with a different database.
As you practice this empathy and self examination, you’ll find yourself writing better code. And if you don’t, well hell, you got tricked into not being an asshole. That’s not so bad, is it?