Essential reads
Books
S poetry
What is it?
A book about using the S language. There are two main dialects of S:
- R (created by the R Project for Statistical Computing)
- S-PLUS (now S+ owned by TIBCO)
Why should I care?
There are other reasons, but perhaps the two best points are:
- There is a glossary of jargon
- There is a chapter on arrays with more than 2 dimensions
Changes since publication
S Poetry was written when R was in its infancy, and version 4 of the S language was not yet publicly available. There have been a number of changes since then.
If you are using R, probably the best way to figure out what is not right in S Poetry is to read about the differences between R and S in the R FAQ (ISBN 3-901167-51-X) on the R Project Home Page .
If you are using S-PLUS, almost everything is still true (except some of the bugs are fixed). However there are some new features that are useful.
The R Inferno
What is it?
Abstract: If you are using R and you think you’re in hell, this is a map for you.
A book about trouble spots, oddities, traps, glitches in R. Many of the same problems are in S+.
Why should I care?
Even if it doesn’t help you with your problem, it might amuse you (and hence distract you from your sorrow).
Getting it
- You can get the R inferno pdf for free here.
Alternatively you can spend an extraordinary amount on the paperback of The R Inferno.
Tao Te Programming
What is it?
Tao Te Programming is a different sort of software book. It steps back from variables and syntax and code details.
If you are looking for “The Tao of Programming”, this isn’t it (hopefully you’ll come to the conclusion that this is better). If you insist, it can be found here and perhaps more officially, here.
Why should I care?
Tao Te Programming tries to get at the spirit of programming, to expose the ways of thinking that make programming challenging and fulfilling rather than too hard and grinding.
The table of contents and some sample chapters
Good programming is often about effective compromise. You can go too far in a good direction. That is why many chapters have opponents — an indication of forces you need to try to balance. Chapters can also have allies that point in a similar direction.
You can read the chapters in order. But if there is much in the book, then that something is unlikely to appear via a linear path. Bouncing around chapters is more in the book’s spirit.
Artwork
The artwork was done with some simple functions in R using its random number generation.
Quotes
Unattributed quotes are from Tao Te Ching.
There are also snippets of creation myths scattered through the book. When you program, you create a world — just as Everything-Maker made a world. Ignore the stories if you like.
— Jon Bentley
We used to live with a cat named Esther. She taught us a proverb we hadn’t known before:
Go for the hand and not the string.
She, for one, believed in direct action.
A graphical user interface is a substitute for programming, sort of. If you are in a restaurant in a foreign country, then pointing to pictures on a card will get you something to eat. You’ll survive. But learning the language will get you farther.
The main reason people don’t want to program is because it is hard. Yes, it’s hard. But not programming may be harder still.
The easy path looks hard.
The hard path looks easy.
Cooks do what they do for survival. Chefs do what they do for joy. A chef works harder than a cook but is happier.
Do you want to be a cook or a chef?
Becoming a chef is a quest.
You wouldn’t think you’ve learned chess once you know how all the pieces move.
You don’t know chess until you can think like a chess player.
This principle is even more true for programming. The important part of becoming a programmer is learning to think like a programmer. You don’t need to know the details of a programming language by heart, you can just look that stuff up.
The treasure is in the structure, not the nails.
Feel Chess
Part of learning to think like a programmer is learning to feel like a programmer. Using and creating software are emotional experiences.
Much as we’d like to think that our rational thinking is the important part, the reality is that emotions dominate.
Being a good programmer is as much about emotional strength as intellectual strength.
Your first brush with programming was most likely a negative experience. Don’t let that define your relationship to it.
Take Time
You didn’t learn to ride a bicycle or a skateboard in one day. You’ll not learn to program in one day. It takes time. Relax.
We may say most aptly that the Analytic Engine weaves algebraical patterns just as the Jacquard-loom weaves flowers and leaves.
— Ada Lovelace
Algebra is an example of abstraction. The symbol x stands not for a specific number but for some number. x is some special part of the numbers.
Words are abstractions. We carve out a piece of reality, separate it from the rest, and name it.
The name that is spoken is not the immortal name
When we speak, we string abstractions together to create a sentence or a paragraph that does what we want. Sometimes we don’t have a proper abstraction — we have to make up a new word. New words are born as slang; some words live past adolescence.
Programming is the same process as speaking a natural language. The “words” are different, the syntax is different, the process is the same. If you can talk, you can program.
A difference is that slang is created much more often in programming.
Repetition is the Cue
Wherever there is repetition, there is an opportunity for abstraction.
If you repeatedly sum up numbers and divide by how many numbers there are (we call that the mean), then you should make an abstraction. Create a routine for that and call it mean.
A programmer’s task is to:
- spot repetitions
- package each into the most appropriate abstraction
Sometimes it is easy to see the abstraction that will work best. Sometimes not.
Compression
You can think of programming as an exercise in compression.
When data are compressed, a code is created so that the actual data can be written more compactly. What’s done once has to stay, but repeated parts can be shrunk. Programming is similar.
Carve and compact.
Ally
- Chapter 9: Verbalize and Nounalize
- Chapter 54: Do Not Repeat Repeat Repeat
Here are four steps for general problem solving (though of course we have programming in mind).
- List the starting ingredients
- State the desired results
- Break the journey from step 1 to step 2 into subproblems
- Put the subproblem solutions together
This is a recursive algorithm — we do the same four steps on each of the subproblems, and on their subproblems.
I’m guessing that this is how almost all problems are solved — mostly subconsciously.
Breaking Up
The hard part is step 3 (but sometimes step 2 is murky). This is another case of abstraction, of carving up reality. There may be multiple possible combinations of subproblems — your task is to create a reasonable combination.
Avoid feeling that you must solve the whole thing in one go. That’s the recipe for being overwhelmed and stuck.
Our natural inclination is to go from beginning to end when breaking up a problem. If that is not bearing fruit, try working backwards from the end.
The more you practice breaking a problem into subproblems, the better you get at it. This is important enough to practice deliberately.
This section should be surrounded by flashing lights. Breaking a problem into pieces seems to be the central block with programming for most people.
Great acts are done by a series of small deeds
Surprise is Good
The act of breaking the problem apart can highlight connections.
If It’s Not Working
Two things to try if you are not getting your problem solved:
- walk
- sleep
Humans evolved as nomads, our brains work best when we are walking.
Sleep allows us to lay down memories, sort possibilities, and have a brain ready for active duty.
Opponent
- Chapter 6: Don’t Solve the Problem
- Chapter 55: Climb Above the Solution
- Chapter 41: Give Up Control
Chapter 6: Don’t Solve the Problem
Chapter 7: Enjoy Confusion
Chapter 8: Procrastinate
Chapter 9: Verbalize and Nounalize
Chapter 10: Pay Attention to Attention
Chapter 11: Be Accident Prone
Chapter 12: Conquer Time
Chapter 13: Learn the Local Jargon
Chapter 14: Accept Numerical Reality
Chapter 15: Be Stateless
Chapter 16: Travel in Space
Chapter 17: Use Your Frustration
Chapter 18: Be a Hacker
Chapter 19: Don’t Crash the Ambulance
Chapter 20: Do Not Plagiarize
Chapter 21: Decontaminate
Chapter 22: Be Miserly
Chapter 23: Curse Knowledge
Chapter 24: Rise to the Occasion
Chapter 25: Be Poetic
Chapter 26: Be Lazy
Chapter 27: Be Impatient
Chapter 28: Have Hubris
Chapter 29: Be Consistent
Chapter 30: Relish Magic
Chapter 31: Tell a Good Story
Chapter 32: Make Bricks Not Monoliths
Chapter 33: Write Opaque Code
Chapter 34: Write Invisible Code
Chapter 35: Engage Eyes
Chapter 36: Grow a Cathedral
Chapter 37: Become a Ghost
Chapter 38: Do Not Be Helpful
Chapter 39: Do Nothing Well
Chapter 40: Find Your Stickiness
Chapter 41: Give Up Control
Chapter 42: Be Quiet
Chapter 43: Give Them Their Own
Chapter 44: Don’t Borrow, Steal
Chapter 45: Always Softcode
Chapter 46: Topple Fences
Chapter 47: Be Claustrophilic
Chapter 48: Be Wary
Chapter 49: Lose Every Battle
Chapter 50: Avoid the Plague
Chapter 51: Comment Quietly
Chapter 52: Beware Longevity
Chapter 53: Beware Easy
Chapter 54: Do Not Repeat Repeat Repeat
Chapter 55: Climb Above the Solution
Chapter 56: Hinder Your Users
Chapter 57: Develop a Sense of Kludge
Chapter 58: Understand Bugs
Chapter 59: Spot Bugs
Chapter 60: Know Why It Works
Chapter 61: Think Safety
Chapter 62: Respect Bug Time
Chapter 63: Dance the Debug 2-Step
Chapter 64: Clean Up After the Flood
Chapter 65: Play
Chapter 66: Sidestep Show Stoppers
Chapter 67: Do a Premortem
Chapter 68: Eat Your Own Cooking
Chapter 69: Bandage Sparingly
Chapter 70: Purge
Chapter 71: Don’t Oversharpen
Chapter 72: Think Backwards
Chapter 73: King Your Users
Chapter 74: Shake Vigorously
Chapter 75: Prove Yourself Wrong
Chapter 76: Sling Garbage
Chapter 77: Tattle on Yourself
Chapter 78: Be a Polyglot
Chapter 79: Avoid Perfect
Chapter 80: Beget Quality
Chapter 81: Follow The Way
My Story