Multithreading?

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

Multithreading?

Heinz Diehl
Hi,

just want to ask if there's a list what programs in opensuse 11.0/11.1 are
multithreaded, meaning "can use multiple cpu cores". Example: "make -j".
Tried to find out by myself but wasn't successful. Can somebody point me to
a side or source of information where I can read more (if it exists)?

--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

Per Jessen-2
Heinz Diehl wrote:

> Hi,
>
> just want to ask if there's a list what programs in opensuse 11.0/11.1
> are multithreaded, meaning "can use multiple cpu cores". Example:
> "make -j". Tried to find out by myself but wasn't successful. Can
> somebody point me to a side or source of information where I can read
> more (if it exists)?

Immediate examples I can think of are:

apache, mysql, postfix.


/Per

--
Per Jessen, Zürich (3.3°C)

--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

Roger Oberholtzer
In reply to this post by Heinz Diehl
On Tue, 2009-02-10 at 17:34 +0100, Heinz Diehl wrote:
> Hi,
>
> just want to ask if there's a list what programs in opensuse 11.0/11.1 are
> multithreaded, meaning "can use multiple cpu cores". Example: "make -j".
> Tried to find out by myself but wasn't successful. Can somebody point me to
> a side or source of information where I can read more (if it exists)?

tcl/Tk /tclsh and wish) are multi-threaded. At least I think they are
compiled that way on openSUSE.

--
Roger Oberholtzer

OPQ Systems / Ramböll RST

Ramböll Sverige AB
Krukmakargatan 21
P.O. Box 17009
SE-104 62 Stockholm, Sweden

Office: Int +46 8-615 60 20
Mobile: Int +46 70-815 1696

--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

gregfreemyer
In reply to this post by Heinz Diehl
On Tue, Feb 10, 2009 at 11:34 AM, Heinz Diehl <[hidden email]> wrote:
> Hi,
>
> just want to ask if there's a list what programs in opensuse 11.0/11.1 are
> multithreaded, meaning "can use multiple cpu cores" . Example: "make -j".
> Tried to find out by myself but wasn't successful. Can somebody point me to
> a side or source of information where I can read more (if it exists)?
>

As you research this you need to understand "multithreaded" does not
equate to "can use multiple cpu cores".

Linux supports several threading models.  I think only "Native POSIX
Linux Threading" can leverage multiple cpus.

OTOH, lots of user space apps utilize multiple processes without the
use of threading and thus they "can use multiple cpu cores".

The make command in your example I believe is an example of a app the
"can use multiple cpu cores", but is not multithreaded.

Apache is another.

In general linux uses lots of small apps doing specific things.  That
is an ideal implementation that "can use multiple cpu cores".

Greg
--
Greg Freemyer
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf

The Norcross Group
The Intersection of Evidence & Technology
http://www.norcrossgroup.com
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

Randall Schulz
In reply to this post by Heinz Diehl
On Tuesday February 10 2009, Heinz Diehl wrote:
> Hi,
>
> just want to ask if there's a list what programs in opensuse
> 11.0/11.1 are multithreaded, meaning "can use multiple cpu cores".
> Example: "make -j". Tried to find out by myself but wasn't
> successful. Can somebody point me to a side or source of information
> where I can read more (if it exists)?

All Java programs will benefit, even if not written for multi-threading
(though any Java GUI has at least one thread devoted / restricted to
handling GUI activity). Even non-GUI Java programs will exploit threads
for I/O and for garbage collection.

Another thing you can do to find answers to your specific question is
use ps or qps to see which programs actually have created multiple
threads.

Lastly, if the question is whether to buy a system with a multi-core
CPU, the answer is almost certainly "yes," since it will continue to be
a more exploited technology over the immediate future.


Randall Schulz
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

David Haller-3
In reply to this post by Heinz Diehl
Hello,

On Tue, 10 Feb 2009, Heinz Diehl wrote:
>just want to ask if there's a list what programs in opensuse 11.0/11.1 are
>multithreaded, meaning "can use multiple cpu cores". Example: "make -j".

make -j just starts multiple programs.

Real multihreading programs link to a threading library. E.g.:

    rpm -q --whatrequires libpthread.so.0

gives you a list of packages linking against the "classic" pthread
library.

HTH,
-dnh

--
"Stef, if you were any stupider you'd be on life support."
                               -- Miranda on userfriendly
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

Randall Schulz
On Tuesday February 10 2009, David Haller wrote:
> Hello,
>
> On Tue, 10 Feb 2009, Heinz Diehl wrote:
> >just want to ask if there's a list what programs in opensuse
> > 11.0/11.1 are multithreaded, meaning "can use multiple cpu cores".
> > Example: "make -j".
>
> make -j just starts multiple programs.

Which is one way to be free of the need to explicitly code for
concurrent execution using threads. If there are multiple processors /
cores, then this approach will cause parallel execution of the
processes that make initiates. Since make understands dependencies
(that's what it's all about), it knows what tasks have no
interdependencies and can thus be executed concurrently.

So use of "make -j" will indeed exploit a multi-CPU / multi-core system.


> Real multihreading programs link to a threading library. E.g.:
>
>     rpm -q --whatrequires libpthread.so.0
>
> gives you a list of packages linking against the "classic" pthread
> library.

But that will not pick up programs such as make that do exploit multiple
CPUs or cores. To wit:

% rpm -q --whatprovides $(type -p make)
make-3.81-128.9

% rpm -q --whatrequires libpthread.so.0 |egrep -i make
[none]


> HTH,
> -dnh


Randall Schulz
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

Randall Schulz
In reply to this post by David Haller-3
On Tuesday February 10 2009, David Haller wrote:
> ...
>
>     rpm -q --whatrequires libpthread.so.0
>
> gives you a list of packages linking against the "classic" pthread
> library.

And to see that this is not a rare thing, try this:

# All pthreads dependencies:
% rpm -q --whatrequires libpthread.so.0 |wc -l
410

# Library pthreads dependencies:
%rpm -q --whatrequires libpthread.so.0 |egrep lib |wc -l
138

# Non-library pthreads dependencies:
% rpm -q --whatrequires libpthread.so.0 |egrep -v lib |wc -l
272



> HTH,
> -dnh


Randall Schulz
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

Wendell Nichols
In reply to this post by gregfreemyer
Greg Freemyer wrote:

> On Tue, Feb 10, 2009 at 11:34 AM, Heinz Diehl <[hidden email]> wrote:
>  
>> Hi,
>>
>> just want to ask if there's a list what programs in opensuse 11.0/11.1 are
>> multithreaded, meaning "can use multiple cpu cores" . Example: "make -j".
>> Tried to find out by myself but wasn't successful. Can somebody point me to
>> a side or source of information where I can read more (if it exists)?
>>
>>    
>
> As you research this you need to understand "multithreaded" does not
> equate to "can use multiple cpu cores".
>
> Linux supports several threading models.  I think only "Native POSIX
> Linux Threading" can leverage multiple cpus.
>
> OTOH, lots of user space apps utilize multiple processes without the
> use of threading and thus they "can use multiple cpu cores".
>
> The make command in your example I believe is an example of a app the
> "can use multiple cpu cores", but is not multithreaded.
>
> Apache is another.
>
> In general linux uses lots of small apps doing specific things.  That
> is an ideal implementation that "can use multiple cpu cores".
>
> Greg
>  
Add to your list:
-almost all java apps more sophisticated than "hello world"
-all your various web and file browsers
-almost anything with a gui will use a MVC architecture where different
threads are responsible for different things

Lots of multithreading going on, but not always lots of GOOD
multithreading.  Poorly written apps routinely synchronize threads badly
(ever have a gui hang and become unresponsive? almost certianly blocked
waiting for a shared resource).

make -j is an example of multiprocessing rather than multithreading as
make synchronizes among many different processes.  In my experience it
doesn't always work well anyway.

Because linux is a multi processing/multithreaded system you can be sure
that for cpu's or cores, the more the better up to a limit.  And that
limit is rising over time... no idea where it is now, but 20+ seems
likely.

wcn
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

Heinz Diehl
In reply to this post by Randall Schulz
On 10.02.2009, Randall R Schulz wrote:

> % rpm -q --whatrequires libpthread.so.0 |wc -l
> 410

> # Library pthreads dependencies:
> %rpm -q --whatrequires libpthread.so.0 |egrep lib |wc -l
> 138
 
> # Non-library pthreads dependencies:
> % rpm -q --whatrequires libpthread.so.0 |egrep -v lib |wc -l
> 272

This is my opensuse 11.0:

liesel:~ # rpm -q --whatrequires libpthread.so.0 |wc -l
42

liesel:~ # rpm -q --whatrequires libpthread.so.0 |egrep lib |wc -l
16

liesel:~ # rpm -q --whatrequires libpthread.so.0 |egrep -v lib |wc -l
26

It seems my quadcore cpu has not enough workload to even exploit one core,
and the other three are just there to consume (a lot of) power. I'll
downgrade to an energy efficient 45W Athlon soon.

Thanks to all who has answered!

--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

Randall Schulz
On Tuesday February 10 2009, Heinz Diehl wrote:
> ...
>
> It seems my quadcore cpu has not enough workload to even exploit one
> core, and the other three are just there to consume (a lot of) power.
> I'll downgrade to an energy efficient 45W Athlon soon.

CPU power consumption is directly tied to instruction execution.
Individual cores consume very little power unless and until they're
actually used and as soon as they become idle, their power consumption
drops drastically.

(It's true that the operating system has to cooperate in this
power-saving behavior, but all of them do.)


> Thanks to all who has answered!


Randall Schul
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

Roger Oberholtzer
In reply to this post by Heinz Diehl
On Tue, 2009-02-10 at 22:12 +0100, Heinz Diehl wrote:

> On 10.02.2009, Randall R Schulz wrote:
>
> > % rpm -q --whatrequires libpthread.so.0 |wc -l
> > 410
>
> > # Library pthreads dependencies:
> > %rpm -q --whatrequires libpthread.so.0 |egrep lib |wc -l
> > 138
>  
> > # Non-library pthreads dependencies:
> > % rpm -q --whatrequires libpthread.so.0 |egrep -v lib |wc -l
> > 272
>
> This is my opensuse 11.0:
>
> liesel:~ # rpm -q --whatrequires libpthread.so.0 |wc -l
> 42
>
> liesel:~ # rpm -q --whatrequires libpthread.so.0 |egrep lib |wc -l
> 16
>
> liesel:~ # rpm -q --whatrequires libpthread.so.0 |egrep -v lib |wc -l
> 26
>

The numbers above and your comment below are really unrelated.

> It seems my quadcore cpu has not enough workload to even exploit one
> core, and the other three are just there to consume (a lot of) power.
> I'll downgrade to an energy efficient 45W Athlon soon.

A threaded app is one where a single app can possibly run in different
cores/CPUs, perhaps even concurrently. The presence of this library
indicates that the app is perhaps creating threads directly, and at
least considering this possible concurrency. For threading to have any
effect, the OS must support multiple cores/CPUs. So threading builds on
and requires multiprocessing.

A multiprocessor system, on the other hand, is managed by the OS. It can
put processes to run in different core/CPU. The applications themselves
need know nothing about this. They do not need to do any threading or
call anything in a thread library. So, vi could run in one core at the
same time as some unrelated process runs in another. And vi need not do
anything to allow this unrelated process to do so. No libpthread is
needed. It is the OS that manages this. If vi wanted to get fancy and do
more than one thing at the same time, it would involve libpthread.

So, the presence of libpthread does not indicate how busy your CPU cores
are. Try 'ps -efLmP' to list all threads. Those listed with the same PID
are all threads of that PID. The PSR column tells which CPU the process
is assigned to. I think that is the column you are interested in. You
will note that processes with one thread still run in different CPUs.

Hope this helps. top also has some thread viewing options. I have a
compute-intensive multithreaded app that tracks the time spent in each
thread. On a 2 CPU processor, I am always happy to see it report that in
30 minutes it used almost 60 minutes of CPU time. Luckily, this app can
be divided into two threads. Adding more CPUs would probably not help it
much. So, YMWV.

Of course, if the system is not busy, there is always the address below.
Anyone finding themselves in possession of a needlessly fast computer
can ship them there. No questions asked. :)

--
Roger Oberholtzer

OPQ Systems / Ramböll RST

Ramböll Sverige AB
Krukmakargatan 21
P.O. Box 17009
SE-104 62 Stockholm, Sweden

Office: Int +46 8-615 60 20
Mobile: Int +46 70-815 1696

--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

make -j (was: Multithreading?)

Per Jessen-2
In reply to this post by Wendell Nichols
Wendell Nichols wrote:

> make -j is an example of multiprocessing rather than multithreading as
> make synchronizes among many different processes. In my experience it
> doesn't always work well anyway.

Different topic, but I use 'make -j' A LOT - several thousand jobs per
hour - in my experience it works really well.  



--
Per Jessen, Zürich (0.1°C)

--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

Larry Stotler
In reply to this post by Heinz Diehl
On Tue, Feb 10, 2009 at 4:12 PM, Heinz Diehl <[hidden email]> wrote:
> It seems my quadcore cpu has not enough workload to even exploit one core,
> and the other three are just there to consume (a lot of) power. I'll
> downgrade to an energy efficient 45W Athlon soon.

One thing to keep in mind is that almost all processors are idle most
of the time in a desktop environment.  Even the P3/700 in my Thinkpad
X21 is idle when it's just sitting there while I am reading a webpage
or typing a reply.  The advantage to SMP is that things are more
responsive.

My first SMP system was  Dual P-Pro/200.  It was always more
responsive than a single system of a similar speed, but it did use
more power.  That's why I turn it off when I'm at work so it's not
wasting power doing nothing.  Since I'm usually gone for at least 10
hours a day, that's a big savings.

Also, for things like doing movie re-encodes, I have found that you
get better performance by running 1 thread per cpu core versus trying
to run 1 encode over multiple cores because the system has to split
the data up and put it back together.  On a quad core system, you can
get 4 movies done faster that way than you can doing 4 one at a time.

Just my 2 cents.
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

Randall Schulz
On Wednesday February 11 2009, Larry Stotler wrote:
> On Tue, Feb 10, 2009 at 4:12 PM, Heinz Diehl <[hidden email]>
wrote:
> > It seems my quadcore cpu has not enough workload to even exploit
> > one core, and the other three are just there to consume (a lot of)
> > power. I'll downgrade to an energy efficient 45W Athlon soon.
>
> One thing to keep in mind is that almost all processors are idle most
> of the time in a desktop environment. ...

That's what SETI@home, Folding@home, BOINC in general and other
distributed collaborative CPU-warmers are for...


> ...
>
> Also, for things like doing movie re-encodes, I have found that you
> get better performance by running 1 thread per cpu core versus trying
> to run 1 encode over multiple cores because the system has to split
> the data up and put it back together.  On a quad core system, you can
> get 4 movies done faster that way than you can doing 4 one at a time.

That's to be expected. Parallelizing an arbitrary computation can yield
anything from no speed-up to near N:1 where N is the number of
independent processors available.

First of all, we must exclude from consideration purely single-threaded
computations which can never exhibit speed-up 'cause they are, by
definition, not parallelizable. (Note that, as mentioned earlier, Java
programs will almost always exhibit some degree of multi-threaded
behavior because at a minimum JVM garbage collection is handled by a
dedicated thread. Presumably other languages or run-times with GC will
do the same.)

For other computations, the degree of benefit obtained by parallelizing
the computation all comes down to a matter of how much the parallel
threads of execution must coordinate with each other and how much they
contend even when not interacting. The former comes when results from
one thread are needed by another. The latter happens when multiple
threads contend for a shared resource, most commonly main store (RAM).

So parallelizing a computation with a great deal of interaction between
the parallel threads will yield much less than N:1 speed-up.
Parallelizing a computation with little or no such interaction and a
very high degree of locality (meaning a high ratio of instructions
executed to cache misses) can exhibit nearly N:1 speed-up.


> Just my 2 cents.


Randall Schulz
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

M Harris-2
On Wednesday 11 February 2009 10:49, Randall R Schulz wrote:
> For other computations, the degree of benefit obtained by parallelizing
> the computation all comes down to a matter of how much the parallel
> threads of execution must coordinate with each other and how much they
> contend even when not interacting. The former comes when results from
> one thread are needed by another. The latter happens when multiple
> threads contend for a shared resource, most commonly main store (RAM).
        The above is really *the* point about smp in general; spot on.

        It is my experience that a well tuned system with one processor (and one
core) runs more efficiently than an smp system with multiple cores that has
not been optimized. The entire system must be optimized for multi-cores, and
the application (whatever that means) must be written specifically with smp
in mind. Throwing multiple cores at an existing system (including apps)
without tuning and optimization can actually result in "slower" performance
(depending)... again, the truth of the paragraph at the top of the page
reigns supreme.






--
Kind regards,

M Harris     <><
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

Larry Stotler
On Wed, Feb 11, 2009 at 1:12 PM, M Harris <[hidden email]> wrote:
>        It is my experience that a well tuned system with one processor (and one
> core) runs more efficiently than an smp system with multiple cores that has
> not been optimized. The entire system must be optimized for multi-cores, and
> the application (whatever that means) must be written specifically with smp
> in mind. Throwing multiple cores at an existing system (including apps)
> without tuning and optimization can actually result in "slower" performance
> (depending)... again, the truth of the paragraph at the top of the page
> reigns supreme.

However, I have found that, at least under linux, that an SMP system
can be faster than a single core system that's 50% faster.  However,
it just depends on what you are doing.  If you are watching a tv show
and surfing the web, then dual core makes much more sense because the
system is more responsive.  So, I don't recommend any single core
systems anymore.  Even the Celerons are almost all DC.  Only the
Sempron is single core anymore.
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

Randall Schulz
On Wednesday February 11 2009, Larry Stotler wrote:
> On Wed, Feb 11, 2009 at 1:12 PM, M Harris <[hidden email]>
wrote:
> > ...
>
> However, I have found that, at least under linux, that an SMP system
> can be faster than a single core system that's 50% faster.  However,
> it just depends on what you are doing.  If you are watching a tv show
> and surfing the web, then dual core makes much more sense because the
> system is more responsive.  So, I don't recommend any single core
> systems anymore.  Even the Celerons are almost all DC.  Only the
> Sempron is single core anymore.

I think that speaks to the fact that there's a good bit of latent
parallelizability at the overall system operation level for what
constitutes typical mixes of activities, both those overtly and
explicitly commanded by the human user and the plethora of
behind-the-scenes processes that interact and coorindate to
provide the user's overall computing experience.


Randall Schulz
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

Larry Stotler
> I think that speaks to the fact that there's a good bit of latent
> parallelizability at the overall system operation level for what
> constitutes typical mixes of activities, both those overtly and
> explicitly commanded by the human user and the plethora of
> behind-the-scenes processes that interact and coorindate to
> provide the user's overall computing experience.

Linux will show good SMP.  With win2k, I only saw about a 10% increase
over 1 proc.  With XP it's a little better.  Haven't really played
with Vista or 7 yet.

I hear the BSDs(free I think) have introduced some really kickin SMP
code that's supposed to be wicked fast.  Haven't had time to play with
that either tho...
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Multithreading?

Larry Stotler
In reply to this post by Randall Schulz
> I think that speaks to the fact that there's a good bit of latent
> parallelizability at the overall system operation level for what
> constitutes typical mixes of activities, both those overtly and
> explicitly commanded by the human user and the plethora of
> behind-the-scenes processes that interact and coorindate to
> provide the user's overall computing experience.

Linux will show good SMP.  With win2k, I only saw about a 10% increase
over 1 proc.  With XP it's a little better.  Haven't really played
with Vista or 7 yet.

I hear the BSDs(free I think) have introduced some really kickin SMP
code that's supposed to be wicked fast.  Haven't had time to play with
that either tho...
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]