Nerd Food: On MinGW, Cygwin and Wine

All operative systems share one feature in common: they all have a kernel. The job of the average kernel is to do low-level stuff like allowing multiple processes to share a physical machine without getting on each others toes. The kernel does this and much, much more, either through its core code or via dynamically loaded modules such as the device drivers provided for your graphics card.

Kernels vary wildly in how they do things internally; that's why there are so many different ones. Even more importantly - at least for our purposes - they greatly differ on how they interface with the layers above them, somewhat derogatorily known as the "user space". The kernel exports an interface to the outside world, but this is normally not meant to be consumed directly; instead users rely on higher-level components to consume it. There are two sides of this interface: the API and the ABI. Unfortunately, very frequently we use API to mean a combination of both, but in reality there is a difference.

When we talk about the API we normally mean a set of headers that can be used to compile code that talks to the kernel directly. For a simple example for the Linux kernel have a look at the Real Time Clock Driver Test/Example Program. You can see it includes a linux kernel header:

#include <stdio.h>
#include <linux/rtc.h>
#include <sys/ioctl.h>
...

These are used to talk directly to the RTC, using standard calls such as ioctl, etc.

From a kernel perspective, its best to think of the ABI as a specification that allows stitching together binary components. The ABI is actually huge, and encompasses very many different aspects, such as:

• The system call ABI, exemplified here for linux. It describes how function calls are setup, how the stack gets cleaned up after the function call, etc.
• The object file format, responsible for describing the layout of binaries such as executables and shared libraries. Most Unices support ELF or the older COFF or the even older a.out; Windows uses PE.

Finally, one last word on internal versus external ABI. Linux tries very hard to make sure the external ABI is kept stable; this is why you can run the same binaries on pretty much any version of the kernel (all things being equal, such as the presence of all dependencies required by the binary). However, Linux makes no guarantees whatsoever with regards to the internal ABI. This means you cannot compile device drivers for a version of the kernel and expect them to work with other versions; they may, but then again they may not. Windows takes the opposite approach, trying hard not to break binary compatibility.

It is somewhat tautological to say that, on UNIX, the outer edge of user space begins with the C POSIX Library. It contains all of what is available on the C Standard Library - that is, the library of functions and types created for the C programming language - and, in addition, it also contains POSIX extensions. It may also contain other non-standard extensions, but we'll ignore these for the sake of simplification.

This entire package is what we commonly call the C Library. If you use Linux you are probably using glibc, the GNU implementation of the C Library. Furthermore:

• When people refer to the C runtime (CRT) they normally mean the shared object (or dynamically linked library) providing the binary implementation of the C Standard Library;
• to link statically against the C runtime means that your application will contain the entire run-time rather than share one with other applications.

Now, its important to bear in mind that there's nothing particularly magical about the C library; is just a set of functions defined by the C programming language and the POSIX standardisation process. Externally it looks like pretty much any other library. What tends to make the C Library a bit special is that pretty much everything on a UNIX system depends on it in one form or another. Also, unlike other libraries, the C Library makes use of kernel APIs to implement the interfaces it exports, and as such it shields the layers above from the vagaries of any particular kernel.

You are probably wondering if these functions would not also be useful to the kernel itself. After all, it is more often than not built in C, right? Unfortunately, due to the highly circular nature of the problem, the kernel cannot make use of the C Library. The kernel uses what is commonly referred to as pure C; that is, only the core C language itself. It must provide its own implementation of all required functions, which can be inspired by code in the C library.

It should be fairly clear by now what the first role of Cygwin is: it aims to provide the missing POSIX API and run time that is required by the majority of applications designed for Linux and other Unices. This it provides via its many DLLs such as cygwin.dll. So all software that runs on Cygwin requires these DLLs - very much like the C Library on Linux is a central dependency.

The second aspect of Cygwin is a bit less obvious. In my quest to explain hows things work I simplified things a bit: POSIX is not only a set of APIs - its also an environment. Yep you heard that right, it is well within the rights of POSIX applications to expect the presence of the Bourne shell (sh) and a host of other utilities such as make and ls.

You may argue that a large number of applications can happily function without bash or ls. Indeed they can - at run-time; however, bear in mind that someone has to build these applications somewhere and it is this very compilation process that tends to require a POSIX environment. Visual Studio does contain fairly capable C and C++ compilers, and one can even access them for free these days using the Express Edition; however, this is a recent development and as such most Linux applications do not have Visual Studio solutions or msbuild files, so its not possible to build them directly from Visual Studio.

The second aspect of Cygwin is then to provide this environment under which to build applications - loosely called a toolchain. As the DLL got closer and closer to POSIX, more and more software has been ported over, making the toolchain closer and closer to Linux; this in turn has made the porting of additional software easier, and the process fed on itself. These days the traditional configure, make and make install are likely to work for a very large number of applications available on a Linux distribution; most of these are the applications shipped with Cygwin, the distribution - its third aspect.

As with everything in life, there's always a cost. You may have heard complaints about Cygwin's performance. Personally I can't moan too much but I guess that if you are running performance sensitive code such as say PostgreSQL under high loads you are likely to notice it. This is for two reasons:

• there is a natural impedance mismatch between the Windows API and POSIX, as described on the previous section; ironing out this mismatch has to cost CPU cycles;
• as a regular Windows denizen, Cygwin has to go via the public Windows API instead of talking directly to the kernel, and as such misses some potential optimisations that Interix/SUA may be using.

Some people are not happy with the dependency on Cygwin, either due to the performance reasons outlined above or due to its fairly viral GPL licence. MinGW tries to plug this gap in the market, providing the following:

• a native port of the GNU toolchain to Windows, including GCC, make, etc. These are full-blown windows applications with no other dependencies;
• the MinGW run-time: headers and lib files that allow compiling against the Microsoft C Standard Library;
• a set of Windows API headers required to compile code against the Windows API.

With these three things one can build native Windows applications that do not rely on POSIX at all (they can, of course, rely on any POSIX functionality Windows may offer directly).

As explained previously, many applications require a POSIX environment on which to build; for instance they may make use of shell scripts so bash is a requirement. To aid in this department, MinGW comes with MSYS, which is effectively an extremely cut-down fork of Cygwin that contains a very minimalist environment with bash, ls and so on. While it may work out of the box for simple use cases, you may find its a bit too basic for non-trivial applications. For instance we found that the version of tar supplied didn't support 32-bit GIDs, causing a lot of spurious warnings. If a basic package like tar, which is trivially used for installing software, must be tinkered with in order to work you can imagine the difficulties in compiling large and complex applications. This is probably not a problem for the typical low-level MinGW user, probably accustomed to embedded development, but it makes it slightly less accessible to the casual developer not interested in the guts of the implementation.

As you probably already guessed, nothing stops you from using MinGW from within Cygwin; in fact, it's available as a package. Doing this gives you the advantages of a full-blown POSIX environment in which to build, rather than the spartan MSYS, whilst still allowing you to generate binaries that do not require Cygwin to run. If you do not want to spend time setting up basic environmental details then Cygwin is the right choice for a key-in-hand solution for a UNIX environment on Windows.

More interesting still, you can run MinGW directly on Linux. Once GCC was taught how to generate binaries for the Windows platform, the next logical step was to allow it to cross-compile these binaries in Linux. This basically means that a MinGW version of GCC is available on your Linux distribution as an ELF binary which is able to generate PE binaries that can be executed on Windows. In this case you won't need MSYS as Linux already provides you with a POSIX environment in which to build. To put things slightly more technically, you can use Linux as both the build and host system, and generate binaries that target Windows.

Finally, a word about MinGW-w64. Originally MinGW only targeted Windows 32-bit - hence why you may see MinGW32 in a lot of places; as 64-bit became more popular, a compiler for it was required - that's where MinGW-w64 came in. These days it provides both 32-bit and 64-bit compilers and it distinguishes itself by both covering more of the Windows API and providing really recent GCC releases. For instance, snapshots of GCC 4.8 are already available from their download site.