This is a special post for quick takes by leogao. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
A 100M LLM is already in the same order of magnitude of Landauer's estimate for the functionnal memory capacity (10^9). But anyway I suppose the question is not about memorizing all the information within your code but just to gain some understanding of the moving parts. If that's so a very small toy model would be enough.
Moreover if we see your python code as an unvompressed version of the LLMs, provided the compression ratio is high in a modern LLM, you have to start with a tiny LLM to finish with a python code of reasonable size.
A preliminary question would be : what is the maximum size of clear python code understable for a competent developer.