No Stupid Questions

It depends on the specifics of how the language is compiled. I'll use C# as an example since that's what I'm currently working with, but the process is different between all of them.

C#, when compiled, actually gets compressed down to what is known as an intermediate language (MSIL for C# specifically). This intermediate file is basically a set of genericized instructions that are not linked to any specific CPU. This is useful because different CPUs require different instructions.

Then, when the program is run, a second compiler known as the JIT (just-in-time) compiler takes the intermediate commands and translates them into something directly relevant to the CPU being used.

When we decompile a C# dll, we're really converting from the intermediate language (generic CPU-agnostic instructions) and translating it back into source code.

To your second point, you are correct that the decompiled version will be more efficient from a processing perspective, but that efficiency comes at the direct cost of being able to easily understand what is happening at a human level. :)

The main issue is that to make code human-readable, we include a lot of conventions that computers don't need. We use specific formatting, name conventions, code structure, comments, etc. to help someone look at the code and understand its function.

Let's say I write code, and I have a function named 'findUserName' that takes a variable 'text' and checks it against a global variable 'userName', to see if the user name is contained in the text, and returns 'true' if so. If I compile and decompile that, the result will be (for example) a function named 'function_002' that takes a variable 'var_local_000' and checks it against 'var_global_115'. Also, my comments will be gone, and finding where the function was called from will be difficult. Yes, you could look at that code and figure out that it's comparing the contents of two variables, but you wouldn't know that var_global_115 is a username, so you'd have to go find where that variable was set and try to puzzle out where it was coming from, and follow that rabbit hole backwards until you eventually find a request for user input which you'd have to use context clues to determine the purpose of. You also wouldn't have the context around what 'var_local_000' represented unless you found where the function was called, and followed a similar line backwards to find the origin of that variable.

It's not that the code you get back from a decompiler is incorrect or inefficient, it's that it's very much not human-readable without a lot of extra investigatory work.

The implicit assumption with decompiling code is that the goal is either to inspect how the code works, or to try compiling for a different machine. I'll try to explain why the latter is quite difficult.

As you said, compilation to machine code only keeps the details needed for the CPU to accomplish what was instructed. And indeed, that is supposed to be efficient to run on that CPU, by reason of being targeted exactly for that CPU. But when decompiling, the resulting code will reflect the specificity to that same CPU. If you then try to compile that code for a different CPU, it will likely work, but will likely be inefficient because the second CPU's unique advantages won't be leveraged.

To use an example, consider how someone might divide two large numbers. Person A learned long division in school, and so takes each number and breaks it down into a series of smaller multiplications and subtractions. Person B learned to do division using a calculator, which just involves entering the two numbers and requesting that they be divided.

Trying to do division by blindly giving Person B that series of multiplications and subtractions to do on the calculator is extremely inefficient because Person B knows how to do division easily. But Person B is following Person A's methods, without knowing that the whole point of this exercise is to just divide the two original numbers. Compilation loses context and intent, which cannot be recovered from decompilation, for non-trivial programs.

Here is an example why source code is useful when it provides context: https://en.m.wikipedia.org/wiki/Fast_inverse_square_root#Overview_of_the_code . Very few people would be able to figure out how this works from just the machine code.