2CPU.com » News » January 2004 » Linux Kernel Hacker Interview @ Arstechnica

Three of the guys over at Arstechnica got together and interviewed Robert Love, a prominent Linux Kernel hacker. The topic of conversation? Version 2.6 of the Linux Kernel, of course.

Ars: What's going on with explicit hyperthreading2 support for Pentium 4? As we understand it, the 2.6 scheduler treats logical processor pairs as independent entities with independent caches and independent functional units. There's a batch scheduler in the works that promises to schedule things with an awareness that resources are shared, as well as scheduling similar-priority threads together. What's planned, ultimately, for this work?

Love: Optimizing for HT

12/02/2003 03:04 PM: Linux Kernel Security Advisory! by Jim
I noticed by way of the The Inquirer that a new security flaw has been found in 2.4.x Linux kernels. It allows local users to escalate their privileges. The problem is that the kernel function "d...

11/07/2001 12:50 AM: New Thunder K7 BIOS, and a New Linux Kernel by Hooz
According to Electic Tech, there is a new BIOS for the Thunder K7. What does it fix? Update MP support, add L1 cache to the total cache size display, add Latency, AGP Fast Write override options.Whil...

05/15/2000 01:56 PM: Linux Kernel configuration by J0rdan
I had a request to go over some kernel configuration/tweaking in the Linux forum. I went through a general workstation configuration of mine that should bring your kernel down to the 400k range. I had...

dualboot_2xcpu Removed from forum Posts: 2396 Joined: 2001-07-16

#26082 Posted on: 01/22/2004 08:53 PM So is the way the linux kernel handles the SMP/SMT hardware efficient? /*asskoala if you reply to this please dont be biased against linux, I really want to know your opinion without all the bias on this matter.*/ alex I finally have SMP. Looking for the next Mrs. Right now (c) 2004. HEMI.

jives BP6 User Posts: 2399 Joined: 2001-05-18

#26083 Posted on: 01/22/2004 09:04 PM Hm makes since the problem making each logical CPU look as if it is a real CPU to the system. At least the Linux community is open about this and willing to see it as a problem that needs work. Any one know how Other OSs handle HT in MultiCPU systems?

i_wolf labhair dom as gaelige Posts: 2034 Joined: 2002-11-19

#26084 Posted on: 01/22/2004 11:32 PM i would love to see some HT benchmarks on and off with this new kernel. That was a good question about the efficiency of the 2.6 solution? I reckon benchmarks (real world) woudl be the only way to tell... Hung like a donkey. Go like a horse!

Jim_ Administrator Posts: 3574 Joined: 2000-03-15

#26085 Posted on: 01/22/2004 11:40 PM Originally posted by i_wolf i would love to see some HT benchmarks on and off with this new kernel. That was a good question about the efficiency of the 2.6 solution? I reckon benchmarks (real world) woudl be the only way to tell... ... and you'll get them when I publish the second half of my HT article in early February. It will have the 3.2GHz P4 and dual 3.2GHz Xeons w/ 1MB L3 cache benchmarked under 2.6. Currently I'm planning on looking at compiling performance, php/apache/mysql, and I'll also use DOTS (Database opensource test suite) which involves MySQL and Java. Of course, that's if I can make it work. It's rather involved. I'm open to other Linux benchmark suggestions as well. [url="http://www.2cpu.com"][size=1]2CPU.com[/url] - Because two are always better than one! [url="http://www.jimkirk.org"]jimkirk.org[/url] - Not a Myth any Longer. Just a Dad.[/size]

AssKoala Anti-Zealot @ GATech Posts: 3302 Joined: 2002-01-02

#26086 Posted on: 01/23/2004 12:03 AM Originally posted by dualboot_2xcpu So is the way the linux kernel handles the SMP/SMT hardware efficient? /*asskoala if you reply to this please dont be biased against linux, I really want to know your opinion without all the bias on this matter.*/ alex I've done enough Kernel Hacking (and reading of documentation) to say no. Compared to the XP/2k3 Kernel Implementation, it looks like an undergrad project. I'm still working on learning all the Windows internals. The 2.6 Scheduler is a lot better than the crap they had in 2.4, though, so I give them credit. Actually, a friend of mine had to take the 2.6 Scheduler and implement it in the 2.4 Kernel as a weekend project in his CS class. I have to take that one next semester... Me Webpage | If you always think like an expert, you'll always be a beginner. | "A handful of knowledgeable people is more effective than an army of fools" -Writing Secure Code, 2nd Ed.

jives BP6 User Posts: 2399 Joined: 2001-05-18

#26087 Posted on: 01/23/2004 12:19 AM Originally posted by Jim_ I'm open to other Linux benchmark suggestions as well. I'll take anything you want to do with compression, file compression (bzip2, gzip, etc), picture compression and/or video compression. The vast majority of tasks that I do that consume any noticeable amount of time normally involves compressing or decompressing data in some form.

jives BP6 User Posts: 2399 Joined: 2001-05-18

#26088 Posted on: 01/23/2004 12:23 AM Originally posted by AssKoala I've done enough Kernel Hacking (and reading of documentation) to say no. Compared to the XP/2k3 Kernel Implementation, it looks like an undergrad project. I'm still working on learning all the Windows internals. So does Windows know the difference between the logical vs the physical and correct for that in the scheduler? Originally posted by AssKoala The 2.6 Scheduler is a lot better than the crap they had in 2.4, though, so I give them credit. It

AssKoala Anti-Zealot @ GATech Posts: 3302 Joined: 2002-01-02

#26089 Posted on: 01/23/2004 12:25 AM Originally posted by jives So does Windows know the difference between the logical vs the physical and correct for that in the scheduler? Of course. XP and Server2003 do. That's one of the major differences between Windows 2000 and Windows XP. Me Webpage | If you always think like an expert, you'll always be a beginner. | "A handful of knowledgeable people is more effective than an army of fools" -Writing Secure Code, 2nd Ed.

i_wolf labhair dom as gaelige Posts: 2034 Joined: 2002-11-19

#26090 Posted on: 01/23/2004 04:25 AM Hi there Jim, Good call on the bench's. I can't wait to see what you got in store. bench's i would like to see.... audio compression: LAME mp3 encoding, Ogg encoding Vid encoding: xvid encoding Linpack compilation .... test how long it takes to compile kde (full) [joke.. this takes ages!!] 3d modelling. .... blender3d Could i ask you to try something actually while you are benchmarking. I presume you will be using the latest GCC to compile the source for the apps you will be using.... while you are doing that (i know its huge extra effort) but would you use Intel C Compiler as well (i think its free for linux for educational purposes... and it doesn't get any more educational than this!)... just to see how good properly optimized threaded code runs with the new kernel. Hung like a donkey. Go like a horse!

Jim_ Administrator Posts: 3574 Joined: 2000-03-15

#26091 Posted on: 01/23/2004 04:28 AM I'll look into it. I can't make any promises as this is a relatively large undertaking, but Hooz and I have been curious about Intel's compiler for quite some time. I'll probably run the numbers initially with GCC and then maybe, if I have time, I'll try out the Intel compiler as well. [url="http://www.2cpu.com"][size=1]2CPU.com[/url] - Because two are always better than one! [url="http://www.jimkirk.org"]jimkirk.org[/url] - Not a Myth any Longer. Just a Dad.[/size]

AssKoala Anti-Zealot @ GATech Posts: 3302 Joined: 2002-01-02

#26092 Posted on: 01/23/2004 06:54 AM Originally posted by Jim_ I'll look into it. I can't make any promises as this is a relatively large undertaking, but Hooz and I have been curious about Intel's compiler for quite some time. I'll probably run the numbers initially with GCC and then maybe, if I have time, I'll try out the Intel compiler as well. If Intel's Linux-version Compiler has all the features of the Windows version, it'll produce some insanely fast code. I'd heard that its still missing some features, though. In Compiler benchmarks (benchmarking the code it compiles, not how long it takes), it blows past GCC, which lags behind MS Visual Studio. Especially with SSE2 compiles (I think in SSE2 Optimized code, it was at least triple Visual Studio and GCC's performance). Of course, GCC is about compiling on anything, not being the fastest. Me Webpage | If you always think like an expert, you'll always be a beginner. | "A handful of knowledgeable people is more effective than an army of fools" -Writing Secure Code, 2nd Ed.

i_wolf labhair dom as gaelige Posts: 2034 Joined: 2002-11-19

#26093 Posted on: 01/23/2004 08:59 AM the only problem is that it breaks compatibility with a LOT of gcc code. particularly it had probs with pre linking code for me on certain apps. But having said that that was on version 7 and im told version 8 is out now and has supposedly resolved this issue?! who knows. Funnily enough, code ive compiled myself in the past has seen more of an improvement on my opteron than xeon rig! who would have thunk it?! There was a discussion in ars before about version 7 ICC and that its heavy use of autovectorised SSE2 even in instances where it shouldn't be used... e.g. where IEEE floating point precision is required has meant that the resulting binary produces erronous results. I don't know enough to comment on that. Just read it yonks ago and it sounded like it should be taken with a grain of salt at the time. but still caught my interest. I think it was koepi or was it nick ... that compiled part of xvid code and said they got a free 10 to 20 percent improvement! (my own memory is a bit dodge) but this is a real impressive improvement. Jim/Hooz it would be awesome if you pull off these benchs! Hung like a donkey. Go like a horse!

Forge labhair dom as gaelige Posts: 720 Joined: 2001-05-12

#26094 Posted on: 01/23/2004 02:09 PM Maybe a little GCC vs. ICC on Opteron, too, if I can swing it. Could get nifty rates of change to graph, then, too, plus GCC32 and GCC64 code versus ICC32 code. No promises, though. I think I'm going to have a full plate for a while. Registered Linux user 82133 (li.org has a short memory)

jrv-austin Registered User Posts: 256 Joined: 2004-01-19

#26095 Posted on: 01/23/2004 04:50 PM Originally posted by AssKoala Of course, GCC is about compiling on anything, not being the fastest. Historically the goal on gcc was to generate "correct" code, which means it does what the source code intends (which may or may not be what the programmer intended!). An important bias in gcc is the need for maintainability. There are a lot of different processors supported, and a change for one processor must not make other processors less reliable. icc's emphasis on the other hand has always been speed of the generated code, and Intel has always been willing to run a lot of risks. I was the gcc x86 maintainer for a few years around 1990 and rejected numerous patches from Intel precisely because of the reliability issue. They'd send a patch, I'd run a test suite and get half a dozen bombs, and that was that. Generally they weren't interested in fixing it the issue and so the change was dropped - it was like they were paid to write code and not to debug it. The other problem is that the x86 violates the design criteria in gcc for the "weakest" processor supported. There are nowhere near enough registers for the reg allocator, and some instructions must use memory in unusual ways (float to int conversions). gcc's architecture was unfortunately set in the 1980s when nobody dreamed an x86 architecture would survive. icc and the Microsoft compilers have had the luxury of only 3-4 processors to consider, and weren't being done on volunteer time. Tyan K8W S2885, 2x Opteron 248, 8GB ECC DDR400, 3ware 8506-8 mirrored 2x Raptor 74, 2x 7k250 HP zx6000 1x 1.5 GHz 6MB Itanic2

2CPU.com » News » January 2004 » Linux Kernel Hacker Interview @ Arstechnica