This is more of a theoretical question than anything. I'm a Comp sci major with a huge interest in low level programming. I love finding out how things work under the hood. My specialization is compiler design.
Anyway, as I'm working on my first compiler, things are occurring to me that are kind of confusing.
When you write a program in C/C++, the traditional thing people know is, a compiler magically turns your C/C++ code into native code for that machine.
But something doesn't add up here. If I compile my C/C++ program targeting the x86 architecture, it would seem that the same program should run on any computer with the same architecture. But that doesn't happen. You need to recompile your code for OS X or Linux or Windows.(And yet again for 32-bit vs 64-bit)
I'm just wondering why this is the case? Don't we target the CPU architecture/instruction set when compiling a C/C++ program? And a Mac OS and a Windows Os can very much be running on the same exact architecture.
(I know Java and similar target a VM or CLR so those don't count)
If I took a best-shot answer at this, I'd say C/C++ must then compile to OS-specific instructions. But every source I read says the compiler targets the machine. So I'm very confused.
No, you are not just targeting a CPU. You are also targeting the OS. Let's say you need to print something to the terminal screen using
cout will eventually wind up calling an API function for the OS the program is running on. That call can, and will, be different for different operating systems, so that means you need to compile the program for each OS so it makes the correct OS calls.
How do you allocate memory? There's no CPU instruction for allocating dynamic memory, you have to ask the OS for the memory. But what are the parameters? How do you invoke the OS?
How do you print output? How do you open a file? How do you set a timer? How do you display a UI? All of these things require requesting services from the OS, and different OSes provide different services with different calls necessary to request them.
Don't we target the CPU architecture/instruction set when compiling a C/C++ program?
No, you don't.
I mean yes, you are compiling for a CPU instruction set. But that's not all compilation is.
Consider the simplest "Hello, world!" program. All it does is call
printf, right? But there's no "printf" instruction set opcode. So... what exactly happens?
Well, that's part of the C standard library. Its
printf function does some processing on the string and parameters, then... displays it. How does that happen? Well, it sends the string to standard out. OK... who controls that?
The operating system. And there's no "standard out" opcode either, so sending a string to standard out involves some form of OS call.
And OS calls are not standardized across operating systems. Pretty much every standard library function that does something you couldn't build on your own in C or C++ is going to talk to the OS to do at least some of its work.
malloc? Memory doesn't belong to you; it belongs to the OS, and you maybe are allowed to have some.
scanf? Standard input doesn't belong to you; it belongs to the OS, and you can maybe read from it. And so on.
Your standard library is built from calls to OS routines. And those OS routines are non-portable, so your standard library implementation is non-portable. So your executable has these non-portable calls in it.
And on top of all of that, different OSs have different ideas of what an "executable" even looks like. An executable isn't just a bunch of opcodes, after all; where do you think all of those constant and pre-initialized
static variables get stored? Different OSs have different ways of starting up an executable, and the structure of the executable is a part of that.
Strictly speaking, you don't need to
You have wine, the WSL1 or darling, which all are loaders for the respective other OS' binary formats. These tools work so well because the machine is basically the same.
When you create an executable, the machine code for "5+3" is basically the same on all x86 based platforms, however there are differences, already mentioned by the other answers, like:
These differ. Now, eg. wine makes Linux understand the WinPE format, and then "simply" runs the machine code as a Linux process (no emulation!). It implements parts of the WinAPI and translates it for Linux. Actually, Windows does pretty much the same thing, as Windows programs do not talk to the Windows Kernel (NT) but the Win32 subsystem... which translates the WinAPI into the NT API. As such, wine is "basically" another WinAPI implementation based on the Linux API.
Also, you can actually compile C into something else than "bare" machine code, like LLVM Byte code or wasm. Projects like GraalVM make it even possible to run C in the Java Virtual Machine: Compile Once, Run Everywhere. There you target another API/ABI/File Format which was intended to be "portable" from the start.
So while the ISA makes up the whole language a CPU can understand, most programs don't only "depend" on the CPU ISA but need the OS to be made work. The toolchain must see to that
Actually, you are rather close to being right, however. You actually could compile for Linux and Win32 with your compiler and perhaps even get the same result -- for a rather narrow definition of "compiler". But when you invoke the compiler like this:
c99 -o foo foo.c
You don't only compile (translate the C code to, eg., assembly) but you do this:
There might be more or less steps, but that's the usual pipeline. And step 2 is, again with a grain of salt, basically the same on every platform. However the preprocessor copies different header files into your compilation unit (step 1) and the linker works completely differently. The actual translation from one language (C) to another (ASM), that is what from a theoretical perspective a compiler does, is platform independent.
For a binary to work properly (or in some cases at all) there are a whole lot of ugly details that need to be consistent/correct including but probablly not limited to.
Differences in one or more of these things are why you can't just take a binary intended for one OS and load it normally on another.
Having said that it is possible to run code intended for one os on another. That is essentially what wine does. It has special translator libraries that translate windows API calls into calls that are available on Linux and a special binary loader that knows how to load both windows and Linux binaries.