I also asked myself the same question a few months ago. After spending a day researching the subject, it seemed like there is almost zero information about the internals of modern phones, and that they are unlikely to have any accessible hardware debug interface. Thus, writing the hardware drivers for such an embedded system will be a very challenging task, if you were to write your own embedded system from scratch.

Perhaps a better take would be to start with getting a custom build of Android to run, and then making changes to the system to fit your needs. Good luck!