Number theory is one of the most elementary parts of mathematics and yet it is also one of the most subtle.
Number theory started by focusing on integers and usually relies on statements regarding their properties. From this perspective, it is one level above set theory and logic. It is a foundation of mathematics as basic as geometry and more basic than algebra.
Pierre de Fermat is usually given credit for being the father of number theory. This is a very interesting attribution since the man was actually a chief magistrate and he almost never published any results. Instead, he focused on posing very challenge math puzzles. It is based on these puzzles and the brilliant solutions that have been proposed by others, Fermat earned his reputation. Fermat also created his factorization algoritgm based on the number theory.
Fermat himself would probably credit Diophantus as the father of number theory. It was a book by Diophantus that first got Fermat interested in the subject and from this perspective, his puzzles were all founded on the ideas of Diophantus.