{"id":73,"date":"2015-01-26T03:37:04","date_gmt":"2015-01-26T03:37:04","guid":{"rendered":"http:\/\/blog.arrozcru.org\/?p=73"},"modified":"2015-01-26T03:37:04","modified_gmt":"2015-01-26T03:37:04","slug":"faster-mame-build-times","status":"publish","type":"post","link":"http:\/\/blog.arrozcru.org\/?p=73","title":{"rendered":"Faster MAME build times"},"content":{"rendered":"

MAME<\/a> is quite a big project. It has\u00a05538 C source files and\u00a03304 header files (counting all 3rd party libraries<\/a>). There are a total of 4416185 lines of code.<\/p>\n

The source files’ extension is c<\/strong>, but the project is actually written in C++. The transition from C to C++ happened a few years ago, mainly led by Aaron Gilles<\/a>. C++ is known[citation needed]<\/sup> to take longer than C to compile.<\/p>\n

The result of all this is that MAME takes a very long time to build. With my Intel Core 2 Quad Q6600 running Ubuntu 14.04, it takes almost 2 hours<\/strong> to build MAME. In this post I will give a few tips on how to get faster MAME build times. Some tips are only available for a few operating systems.<\/p>\n

1. Use clang<\/a> instead of gcc<\/a><\/h2>\n

GCC is awesome. It’s the compiler that powers the open-source world. It’s been doing this for over 27 years. But some 11 years ago LLVM<\/a> came along, and some 7 years ago came clang, now a serious contender in the compilers world.<\/p>\n

You can specify the compiler you want to use while building MAME with the CC=<compiler><\/code> option.<\/p>\n

$ time make CC=clang\r\n[...]\r\nreal    72m44.722s\r\nuser    65m47.202s\r\nsys 4m57.559s\r\n<\/pre>\n
If you don’t want make to print out all compilation commands, just use the @<\/code> sign before the compiler name, like this: make CC=@clang<\/code><\/p>\n
How much of a speedup do we get by using clang instead of gcc? Let’s find out.<\/p>\n
$ time make CC=gcc\r\n[...]\r\nreal    111m37.837s\r\nuser    102m32.533s\r\nsys 8m31.289s<\/pre>\n
The speedup for a full build of MAME is 35%<\/strong> when using clang instead of gcc.<\/p>\n
Clang is now the default compiler in Mac OS X, so there’s no need to specify it in the command line. Unfortunately, clang still does not support Windows.<\/p>\n
2. Use multiple cores<\/h2>\n
With most processors now having multiple cores it’s quite straight-forward that we should be using all those cores for compilation. The compiler itself is not optimized for multiple cores, but since we have a bunch of files to compile and they’re independent of each other, we can compile them in parallel.<\/p>\n
You can specify the amount of jobs to\u00a0run in parallel\u00a0with the -j <number of cores><\/code> option.<\/p>\n
With GCC and two cores we get the compilation time cut almost by half:<\/p>\n
$ time make -j2\r\n[...]\r\nreal    54m45.346s\r\nuser    98m30.421s\r\nsys 8m37.893s\r\n<\/pre>\n
Use as many cores as you can for compilation. It may even be worth using more jobs in parallel than actual number of cores. A good rule is to use <number of cores + 1> jobs.<\/p>\n
3. Disable GNU make builtin rules<\/h2>\n
Up to now we’ve covered speedups for building the whole project. But what if you’re hacking away in MAME, and you have to compile your code over and over with small changes in between each run? The 30 seconds it takes to compile that one change and link MAME seem like FOREVER<\/strong>. Every second that can be scraped off is welcome.<\/p>\n
Every time you run GNU\u00a0make<\/strong>, it will check for all the files that have to be recompiled. If you have changed only one file, make will still check for all other files to see if they have to be recompiled. This is normally quite quick, but GNU make has a thing called implicit rules<\/strong>. It will check for a bunch of other files that you never asked for in the first place.\u00a0I don’t know when this is really useful, but most modern Makefiles don’t need to use any implicit rules. MAME doesn’t.<\/p>\n
You can disable this feature\u00a0with the -r<\/code>\u00a0option.<\/p>\n
To show the benefits of disabling implicit rules, I’ll run make in a fully built directory. Everything has already been built, so there’s nothing for make to do, except for checking if it needs to make any more rules, implicit or explicit.<\/p>\n
On a system with no files cached (cache cleared between runs):<\/p>\n
$ time make\r\nmake: Nothing to be done for `default'.\r\n\r\nreal    0m2.136s\r\nuser    0m0.577s\r\nsys 0m0.120s\r\n\r\n$ time make -r\r\nmake: Nothing to be done for `default'.\r\n\r\nreal    0m0.736s\r\nuser    0m0.119s\r\nsys 0m0.122s<\/pre>\n
When\u00a0all files already in cache:<\/p>\n
$ time make\r\nmake: Nothing to be done for `default'.\r\n\r\nreal    0m0.550s\r\nuser    0m0.504s\r\nsys 0m0.046s\r\n\r\n$ time make -r\r\nmake: Nothing to be done for `default'.\r\n\r\nreal    0m0.128s\r\nuser    0m0.086s\r\nsys 0m0.042s\r\n<\/pre>\n
The gains are very small on most systems (Linux and Mac OS X), being less than 2 seconds in the worst-case scenario. But let’s try this on Windows now:<\/p>\n
> timer make\r\nmake: Nothing to be done for `default'.\r\n\r\ntime taken: 4164 ms\r\n\r\n> timer make -r\r\nmake: Nothing to be done for `default'.\r\n\r\ntime taken: 472 ms\r\n<\/pre>\n
Did you see that? Instead of taking 4.164 seconds<\/strong>, make now takes only 472 milliseconds<\/strong>. The\u00a0gains are HUGE<\/strong> in Windows systems, where file system operations take an awkwardly long amount of time.<\/p>\n
4. Use gold<\/a><\/h2>\n
Suppose we’re still hacking MAME and making small changes in one source file only. Even if we have to compile only one file, we still have to link the MAME executable in its entirety. This means walking through all compiled object files to make one final executable. This step can take considerably longer than compiling any source files that have changed.<\/p>\n
GNU binutils<\/a> has a new linker optimized for ELF files and big\u00a0C++ projects since 2008. This linker is called gold<\/strong>. It does a very good job with MAME.<\/p>\n
You can specify the linker you want to use while building MAME with the LD=<linker><\/code> option.<\/p>\n
You don’t use LD=gold<\/code> directly, but specify that you want g++ to use gold while linking. The command thus becomes:\u00a0LD=\"g++ -fuse-ld=gold\"<\/code><\/p>\n
Let’s see how much speedup we can get with gold instead of the default linker:<\/p>\n
$ rm -f mame64 && time make -r\r\nLinking mame64...\r\n\r\nreal    0m20.442s\r\nuser    0m16.478s\r\nsys 0m2.757s\r\n\r\n$ rm -f mame64 && time make LD=\"@g++ -fuse-ld=gold\" -r\r\nLinking mame64...\r\n\r\nreal    0m5.012s\r\nuser    0m4.185s\r\nsys 0m0.781s<\/pre>\n
The linking step is 75% faster when using gold<\/strong>.<\/p>\n
Unfortunately this linker only works for generating ELF files<\/a>, which means it only works for Linux builds. Mac OS X and Windows can’t use this linker.<\/p>\n
Putting it all together<\/h2>\n
So, different Operating Systems have different tricks to speedup MAME build times.<\/p>\n
For Linux, use clang and gold:
\n$ make -r CC=@clang LD=\"@g++ -fuse-ld=gold\" -j4<\/code><\/p>\n
For Mac OS X, clang is used by default, and you can’t use gold:
\n$ make -r -j4<\/code><\/p>\n
For Windows, you can’t use clang or gold, but at least you can use multiple cores and shave a few seconds off\u00a0by disabling implicit rules:
\n> make -r -j4<\/code><\/p>\n
 <\/p>\n","protected":false},"excerpt":{"rendered":"
MAME is quite a big project. It has\u00a05538 C source files and\u00a03304 header files (counting all 3rd party libraries). There are a total of 4416185 lines of code. The source files’ extension is c, but the project is actually written in C++. The transition from C to C++ happened a few years ago, mainly led … Continue reading Faster MAME build times<\/span> →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-73","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"http:\/\/blog.arrozcru.org\/index.php?rest_route=\/wp\/v2\/posts\/73","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/blog.arrozcru.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/blog.arrozcru.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/blog.arrozcru.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/blog.arrozcru.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=73"}],"version-history":[{"count":4,"href":"http:\/\/blog.arrozcru.org\/index.php?rest_route=\/wp\/v2\/posts\/73\/revisions"}],"predecessor-version":[{"id":78,"href":"http:\/\/blog.arrozcru.org\/index.php?rest_route=\/wp\/v2\/posts\/73\/revisions\/78"}],"wp:attachment":[{"href":"http:\/\/blog.arrozcru.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=73"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/blog.arrozcru.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=73"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/blog.arrozcru.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=73"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}