This is so cool!!!! I'd love to know more! I wish to go on a journey of self-teaching math too, maths is pure... and fun. Could you tell me how maths comes into application in cryptology?
Yeah, I don't know if you feel the same way about math, but to me it's just like solving a puzzle, which is fun for me. :D
So, for cryptography... The two biggest branches of mathematics used are Abstract Algebra and Number Theory. A big part of an encryption system's security often relies on an unsolved problem in Number Theory. A prime (lol) example of this is the RSA cryptosystem. RSA public keys are the product of two very large prime numbers. If an attacker could get their hands on these two prime numbers, they can likely derive the private key from them, but the security relies on the fact that there is currently no known way to factor one of these large products in any reasonable amount of time. While it could be possible some day to find a better way to factor these numbers, many researchers have attempted this with no luck. This is actually kind of good, because if someone does find a way it would likely turn a lot of security on it's head, and people will be running around trying to find a system that could replace RSA.
As for cryptanalysis, the same subjects, but with the addition of Statistics. It's a mistake to think that cryptanalysis involves some bad guys in some dark warehouse somewhere trying to crack codes for malicious purposes. While something like that does happen, especially with adversaries of a government, more often than not it is for the purpose of
building security. It's not too common of knowledge that just because someone spends years researching a new cryptographic algorithm, it's not automatically secure. Instead, that is done, but then the system is made public. Then, and only then, if no blaring weaknesses are found after several years of many dozens of researchers throwing everything they can at it, then it can be deemed secure. This was the case with DES, RSA, and AES, and that process is called the 'peer review process'. The only reason DES is no longer secure is that the performance of technology has outgrown it's small key size. AES has and probably will remain the standard symmetric cipher for a good while longer. RSA has a nice feature of being able to just increase the key size as technology improves, letting it live even longer. I've made some of my own encryption systems over the years, but due to my lack of knowledge, I wouldn't trust it for anything past encrypting your 'secret' recipes. They probably aren't even worth it for a researcher to spend time cryptanalyzing, because they were made by someone who still doesn't have all the knowledge to make one properly. If you are at all curious about this project, however, here is a link to the github repo for it:
ANGELITA_128 Encryption System In that repo is also a 'paper' I wrote on it given my analysis, but again it doesn't amount to much since I barely knew what I was doing. :P
Finally, if you want to self-study, here is my suggestion. Be honest with yourself, determine what you know you know, and what you know you have trouble with. Start at the lowest subjects you have trouble with, then work your way up from there. It helps a lot to set small, realistic goals for what you want to accomplish with it. There may come a time where you are like "I have no idea what I'm doing, and I understand 0% of what I'm looking at! (0_o)". That is pretty normal, some advice I've heard on that is "Don't panic!". I've been there myself, and I ended up just having to slow down and read into several sources on the subject before getting some grasp on it. You can easily find full courses on YouTube, if that helps you learn.
Damn, lol, talk about TLDR, I guess I get carried away when I'm passionate about a subject.