Understanding RNNs and LSTMs: What's the Deal?

What Are They? RNNs are like learners with short-term memory – they remember recent information but get fuzzy as time passes. Simply put, a Recurrent Neural Network (RNN) is like a “learner” with memory. Imagine this learner reading an article, remembering the meaning of each word temporarily before moving on to the next. While traditional neural networks treat each input independently (like starting from scratch each time), RNNs “remember” previous information (earlier words) and pass this memory to the next step, creating context for the entire sentence....

September 20, 2024 · 6 min

Understanding Cross-Entropy

What is it? In deep learning, “cross-entropy” is a function used to measure the difference between two probability distributions, typically used in classification tasks. Analogy: Guessing the Color of Candies Imagine you have a bag of candies containing red, green, and blue colors. You guess the color distribution of the candies is 50% red, 30% green, and 20% blue. However, the actual distribution is 60% red, 20% green, and 20% blue....

September 18, 2024 · 3 min

Why Did My Nix Disappear After Upgrading to macOS 15?!

What’s Going On? Yesterday, I happily upgraded my macOS to 15, but today when I tried to build my site with hugo, I found that hugo was missing?! 1 2 (base) BdimMacBook-Pro:~ bdim404$ hugo -bash: hugo: command not found I thought to myself, I didn’t uninstall hugo! I installed hugo using nixpkgs, so I checked if nix was still there. 1 2 (base) BdimMacBook-Pro:~ bdim404$ nix -bash: nix: command not found Oh no, nix is gone?...

September 17, 2024 · 1 min

Understanding Embedding Layers

Imagine you’re in a classroom with many students, each with their own name. The teacher wants to assign tasks based on students’ names, but the names themselves are meaningless and can’t help the teacher make decisions directly. So, the teacher assigns each student a number, like Xiaoming is No. 1 and Xiaohong is No. 2. These numbers act as “labels” for the students, helping the teacher organize them better. However, the numbers alone aren’t enough, as they don’t carry much information....

September 13, 2024 · 4 min

OMR Related Paper Review: Practical End-to-End Optical Music Recognition for Pianoform Music

Core Ideas 1. Background of the Study: This paper explores how to implement an end-to-end Optical Music Recognition (OMR) system using deep learning methods, with a focus on recognizing pianoform music. While recent progress has been made in monophonic music recognition, existing OMR models struggle to handle the multi-voice and multi-staff nature of piano music. The complexity of piano music stems from its independent parallel voices, which can freely appear and disappear within a composition....

August 27, 2024 · 7 min