Kyle Dotterrer

Tagged “Ai”

  1. Next Token Prediction with MLP

    Extending makemore with an MLP, and running some experiments to minimize validation loss

  2. A Neural Character Bigram Language Model

    A neural character-level bigram language model, optimized with gradient descent rather than counted from the training data.

  3. N-Grams and Other Experiments

    Generalizing the statistical bigram model to arbitrary n-grams, and using it as a vehicle to explore hyperparameter tuning.

  4. A Statistical Character Bigram Language Model

    Building a statistical character-level bigram language model - the first step from raw autograd toward models that emulate natural language.

  5. More Autograd: Arithmetic Operations, Nonlinearities, and a Simple Neural Network Library

    Extending the autograd engine with more arithmetic operations and nonlinear activations, then building a small neural network library on top of it.

  6. A Scalar-Valued Autograd Engine

    Implementing a scalar-valued automatic differentiation engine from scratch. First post following Karpathy's Neural Networks: Zero to Hero.

  7. In Defense of Superintelligence

    A response to Cal Newport's case against superintelligence, laying out where his rebuttal of Yudkowsky's existential-risk argument goes wrong.