RESOURCES

Data Oriented Design and C++ - Mike Acton

When programming, it's often easy to forget the trivial fact that programs exist to transform data. This video was a great reminder of how to actually take advantage of the compiler and hardware (via the memory hierarchy) to improve performance.

Practical Data Oriented Design - Andrew Kelley

Similar to the Acton's 2014 CppCon talk, except it actually goes through best-practices to help achieve DOD.

Statistics

• For probability, most courses are a reiteration of a lower level course but with a more mathematically rigorous framework. For introductory probability, I would recommend Probability by Jim Pitman or Harvard's Stat 110 taught by Joseph Blitzstein. If you're also interested in a non-measure theoretic next level course, I would do some version of stochastic processes, of which I would recommend Durrett's Essentials of Stochastic Processes. Stochastic processes show up everywhere from physics, finance, to reinforcement learning so I'd highly recommend giving that book a read!

  For measure-theoretic probability, my favorite text unfortunately is not yet published. As such, I would recommend three different books. The standard text of "graduate" probability is usually Durrett's Probability: Theory and Examples. If you want a finance-oriented book, I would recommend Michael Steele's Stochastic Calculus and Financial Applications. Finally, my favorite book (other than the unpublished book) is Varadhan's Probability.

• For machine learning and deep learning, I would recommend the two texts An Introduction to Statistical Learning and Elements of Statistical Learning. I believe these books are paired with lectures online by the authors and the second text is often referred to as the key reference text for standard machine learning methods. As for deep learning and reinforcement learning, there are a plethora of publicly available courses at both UC Berkeley and Stanford University which are excellent (CS 182 - Deep Learning, CS 285 - Deep Reinforcement Learning, CS224N - NLP with Deep Learning, CS231n - CNNs for Visual Recognition, PyTorch for Deep Learning, to list a few). I have not worked through all of these resources myself, but I would highly recommend CS 182. As for reinforcement learning, I've heard good thing about Reinforcement Learning, An Introduction by Sutton and Barto which I plan on reading later this year. I'll update my review as necessary.
• Finally, let's talk statistical inference. I would recommend Statistical Inference by Casella and Berger as the standard introductory text for the field. Additionally, I really liked Computer Age Statistical Inference, which I think highlights what "data science" as a field should be. If you want to explore areas like causal inference, I really liked Peng Ding's A First Course in Causal Inference.

Mathematics

• Analysis: in my opinion the best texts is the Stein tetralogy, four books that encompass the Princeton Lectures in Analysis. If you go through all four books (even skipping sections you don't care about), you will be well-versed in undergraduate mathematical analysis. On the more "optimization" end, I would highly recommend Convex Optimization by Boyd and Vandenberghe, which is an essential for any machine learning/deep learning. I believe that online lectures should also be provided by Stanford University.

Computer Science

• For general computer science, I would recommend blogs (like some listed below), my post on teaching myself computer science, or Oz Nova's teachyourselfcs.

My Favorite Blogs/Channels

This is a non-exhausting list of some of my favorite blogs and Youtube channels. This will likely grow as I keep developing my blog!

• Statistical Modeling, Causal Inference, and Social Science is my favorite statistics blog.
• Phrack Magazine has a bunch of great articles for computer security + hacking.
• Keith Conrad's Expository Papers are fun to read if you ever want to learn nuggets of mathematics.
• matklad has a bunch of great resources and I was personally inspired by his "links" section.
• Andrew Kelley's blog for Zig related posts
• Loris Cro's blog. Also for Zig related posts.
• Mitchell Hashimoto's Blog which documents his journey creating Ghostty with Zig.
• research!rsc: Russ Cox's blog.
• Julia Evans' Blog: contains so many good references on just about everything computer science.
• Michael F. Bryan has written about interested Rust patterns.
• Logan Smith has made great videos about Rust internals.
• Ian Henry's Blog has a great guide on Nix and showcases a bunch of interesting projects.
• Kay Lack has super interesting about TCS and low-level coding.