- the idea of semignosis: the struggle when you don’t understand a concept, and the necessity of not giving up, of pushing through it
- when you make a mistake and you identify it, use a failure autopsy to attempt to rewire your neural circuits correctly
- the ultraworking pentathlon: an extended pomodoro with self-reflective questions at every break
- the importance of planning less than you think you can do, since you suffer from optimism bias and will naturally think you can do more than you can anyway
- fundamentals need to be mastered, and devoting regular time during the challenge as ‘fundamentals mastery’ is likely to be very high return
- emphasise practical over theory. Iterate between theory and practical.
- not from the book, but test your knowledge constantly and see how hard you can make your practice sessions.
- when things are hard: if this is all it takes to break me, I would never have been worthy of greatness anyway.
Two key ideas are failure autopsies and insight mining.
Failure autopsy: going over a scenario where you made mistakes and visualising yourself making the correct choice. For example, coding up a reinforcement learning exercise, this would involve:
- writing out how you modelled the problem initially
- writing out how you got to your first solution
- writing out how you got to the best solution (e.g. after looking at others)
- visualise yourself moving straight to the optimal solution
- generalise the scenarios: find other situations where the error might come up and how you would respond to it
Insight mining: When you see a neat trick (like a mathematical jump, or a nice python snippet) write it down and imagine yourself using it in future situations.
The author also talks about managing himself when he is frustrated, using phrases he repeats to himself. I quite like this idea. Taking a break might have a similar effect.
Chapter 2 + 3
Not much here that I want to incorporate. Talks about mantras as a way of constantly reminding yourself of your goals, almost religiously. I think the concept of a mantra stack is quite interesting but I am not going to include this in my projects.
References Josh Watziken’s Art of Learning and how mastering the fundamentals of a field brings great success. It made me think about what the fundamentals are for data science. It isn’t an easy question because the field is quite cross-diciplinary. The fundamentals could refer to data cleaning, R/Python programming, calculus, linear algebra, the basic few machine learning algorithms or other things. I guess each of those is in turn made up of sub-skills, and mastering those gives the benefits. For example mastering the fundamentals of linear algebra would include dot product, cross product, matrix multiplication, eigenvalues, eigenvectors, matrix decomposition and others.
A variant on the pomodoro method is proposed, called an “ultraworking pentathlon”. You work 30 minutes on, 10 minutes off, and do this five times in a row. Before each cycle you ask yourself the following questions:
- What do I plan on accomplishing?
- How will I begin?
- What hazards are present? (this is strange, maybe applies more to physical work?)
- What is my energy and morale like?
Now you do the 30 minutes, and at the end ask:
- Did I accomplish my goal?
- Were there any hazards present?
- How will I improve for my next cycle?
- What is my energy and morale like?
These questions shouldn’t take that long – maybe about a minute or so.
At the beginning of the pentathlon, in addition, ask yourself these questions:
- What’s my first priority today?
- Why is this important for me?
- How will I know when I’m finished?
- Any dangers present? (e.g. procrastination?)
- How many cycles do I think I need for this goal?
- Is my goal concrete or subjective?
At the end, reflect back by asking:
- What did I get done this session?
- How does this compare to my normal output?
- Where did I consistently get bogged down? Is this part of a pattern?
- What big improvement can I make in future cycles?
These questions form a wrapper around the standard pomodoro technique. They can help identify weak spots in your thinking and areas of improvement. If you are trying out a new technique, it lets you evaluate its effectiveness and stay on track to actually use the technique. Write down the results and you can quickly see what is holding you back.
This chapter talks about a concept the author terms ‘semignosis’. This is the state where you are learning something hard but don’t understand it yet. It’s a frustrating state to be in but necessary to understand some things. Eventually things will click and you’ll see the light but until that happens, things are frustrating. Some examples:
- understanding backpropogation
- following a proof you don’t quite understand.
These chapters talk about the process of making the STEMpunk plan and how reality went in comparison to the plan. The plan that Trent came up with was detailed, with resources described at each step. He chose to iterate between theory and practice, reasoning that is the best way to learn and that it’s the natural approach that we learn as kids.
The plan turned out to be far too ambitious. I also felt like he tackled a breadth of topics so wide that he couldn’t focus on anything in depth. If the computing module had only been one of two in the project then Trent could have got a much more deep understanding of computers than otherwise.
Another interesting thing is that his plan didn’t really have prerequisites. The topics are more or less self contained. By contrast, focusing on a topic with a lot of prerequsites like deep reinforcement learning will need a strong understanding of many subskills: neural networks, linear algebra, calculus, probability theory, not to mention programming.
I think the key takeaways here are: - make a detailed plan, but be prepared to revise it during the challenge as it inevitably falls apart - emphasise practical applications of the theory that you learn. Cycle between theory and practice but err on the side of too many practical applications than too few, like trying different algorithms in openai gym games, coding up algorithms from scratch, making a system that uses the algorithms, comparing the performance of the algorithms, where they work and don’t work, common pitfalls and also trying to speed them up.
There’s a great quote here, to say when you are focusing on something difficult and feel like giving up:
If this is all it takes to break me, I would never have been worthy of greatness anyway.