Theory

In the context of your thesis report, theory pertains to general, typically academic knowledge relevant to your report. As a rule of thumb: if you find a topic as a main section in a book required for some other course in your studies, then it is probably suitable to refer to it as theory. Examples are communication protocols, which you will find in networking course literature, database technology (e.g. SQL), which you can find complete academic courses on, operating system design, cryptography, etc.

Of particular interest might be the rare paper Exploit Programming: From Buffer Overflows to “Weird Machines” and Theory of Computation by Sergey Bratus et al., which theorizes exploit development.