Why does it matter? If you want to read scheduling code of linux, no one forces you to read 5M+ lines of driver code. If linux was just a microkernel, and distros shipped drivers, you'd still have to read roughly the same number of lines to understand linux scheduling. If you want to read the entire linux code, you can still do it, just ignore the driver code if you're not interested in them; otherwise in your scenario you'd be reading driver code in distros as well.
We literally read basically all of the Linux scheduler/syscall code in my computer engineering classes in the first couple weeks, the rest of the course was to reimplement a subset of it for a custom (RT)OS on some Cortex-M3 microcontroller dev boards.
The first week or so of the process was learning how to go from a new Debian machine (with the expectation we'd only used Windows or Macs before) through "git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git" (not on Github at the time) to finding the function called by "man 2 fork" and navigating the source tree/unmasking preprocessor stuff to show the actual implementation. Yes, you can easily make a wrong turn if you're trying to do that on your own. But the actual linux/kernel directory, which has most of the parts you'd want, isn't that much larger, and a lot of the difference is modern requirements like power saving and security.
