why /var/run and /run these two directories have absolutely the same contents?

classic Classic list List threaded Threaded
71 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

why /var/run and /run these two directories have absolutely the same contents?

bruce
i know /run is a new cross-distribution location for the storage of transient
state files, and /var/run is replaced by /run. but
1, why /var/run still have contents just as the same as /run ? just for
backward compatibilty ?

2, if a program want to write some data to /run, is it supposed to write the
same data to /var/run ?

3, it seems /var/run is not a symlink to /run, so /var/run occupy harddisk
space, just like  /run does?
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

gregfreemyer


bruce <[hidden email]> wrote:

>i know /run is a new cross-distribution location for the storage of
>transient
>state files, and /var/run is replaced by /run. but
>1, why /var/run still have contents just as the same as /run ? just for
>
>backward compatibilty ?
>
>2, if a program want to write some data to /run, is it supposed to
>write the
>same data to /var/run ?
>
>3, it seems /var/run is not a symlink to /run, so /var/run occupy
>harddisk
>space, just like  /run does?

Bruce, i'm not sitting in front of my machine, but i'm pretty sure it is bind mount.

You should be able see that by looking at the output of "mount".  A bind mount is similar to a symlink.  I don't know the pros/cons of a bind mount vs a symlink.

Greg

--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

Anton Aylward-2
In reply to this post by bruce
bruce said the following on 05/06/2013 08:01 AM:

> i know /run is a new cross-distribution location for the storage of transient
> state files, and /var/run is replaced by /run. but
> 1, why /var/run still have contents just as the same as /run ? just for
> backward compatibilty ?
>
> 2, if a program want to write some data to /run, is it supposed to write the
> same data to /var/run ?
>
> 3, it seems /var/run is not a symlink to /run, so /var/run occupy harddisk
> space, just like  /run does?

You may be missing something, but I'm not sure what.
Try running

    ls -li /run /var/run

That will list the inodes as well as the dates and times and sizes.

Oh look! They are the same in both directories.
What can that mean?
That they are the same files?

How is that possible without a symlink?


Well when I run

# ls -ldi /run /var/run
2377 drwxr-xr-x 22 root root 720 May  6 05:17 /run
2377 drwxr-xr-x 22 root root 720 May  6 05:17 /var/run

Oh, look, they are the same inode.

This is a 12.3 system
Hmm
# mount | grep run
tmpfs on /run type tmpfs (rw,nosuid,nodev,relatime,mode=755)
tmpfs on /var/run type tmpfs (rw,nosuid,nodev,relatime,mode=755)

No, I don't think /var/run is occupying hard disk space just like /run.


One Upon a time someone wrote a file system driver for the V7 PFDPD-11,
I think it was David Tilson, that migrated the first few blocks of the
disk, that meant the superblock and inodes, into memory but left the
data on disk.  In that context, pre networking pre FFS, memory maxing
out at 4Meg, it offered an improvement for the roll-in/roll-out
applications like compilers that made heavy use of temp files.

Now we have virtual memory, much larger memory, inode caching, name path
caching and very fast file systems.  The arguments for tmpfs have a lot
to do with volatility.  In effect these file systems are not very
active, not like the compiler intermediate files that Dave Tilson was
trying to optimise for.  The tmpfs makes use of swap; if this memory is
not accessed it get on the page-out queue.  Its never going to take up
space on the file system part of your disk.

The "if a program want to write some data to /run" should not be taken
in a wide context.  As I said, this isn't a kind of buffer memory,
compiler intermediate pass files, scratchpads and so forth.  Looks
what's there: pid files.  How often will they get accessed?  Sockets.
Dynamically created unit files.   How many of the files there are ovre
few hundred bytes?

Its all very constrained.

If you want temporary files as in buffer files, working space,
intermediate sort files, use /tmp --- oh and clean up after yourself!





--
"Most victories came from instantly exploiting your enemy's stupid
mistakes, and not from any particular brilliance in your own plan."
  -- Orson Scott Card,
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

Cristian Rodríguez-2
El 06/05/13 08:38, Anton Aylward escribió:

  Looks
> what's there: pid files.

pid files there may be a leftover, while systemd supports pidfiles, it
is not the prefered way to track the process number.. the daemons
creating them, 99% of the time do it in a racy fashion; reason being, no
standard api to create pidfiles in libc, yay! (freebsd has pidfile_*()
api though) so a workaround kicks in, namely an inotify watch is placed
on the pidfile for a period of seconds, if the pidfile never appears the
service is later marked as "broken".


--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

L A Walsh
In reply to this post by Anton Aylward-2
Anton Aylward wrote:

> bruce said the following on 05/06/2013 08:01 AM:
>> i know /run is a new cross-distribution location for the storage of transient
>> state files, and /var/run is replaced by /run. but
>> 1, why /var/run still have contents just as the same as /run ? just for
>> backward compatibilty ?
>> 3, it seems /var/run is not a symlink to /run, so /var/run occupy harddisk
>> space, just like  /run does?
>
> You may be missing something, but I'm not sure what.
> Try running
>
>     ls -li /run /var/run
>

The "tmpfs" is mounted on both /run and /var/run  --
so it is the same memory based file system mounted at both points.

That's why you see duplicates in both places -- no symlinks are needed.

A mount --bind would do similar, but it's like procfs -- it gets mounted in
your root -- but also in every root of every "chroot" instance... but
none of them take actual disk space -- just system memory! (which is a darn
good reason not to move /tmp to being RAM based, as it gets used
for large files sometimes, and .. oops. there goes your memory!...;-)


--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

Anton Aylward-2
Linda Walsh said the following on 05/06/2013 09:18 PM:
> (which is a darn
> good reason not to move /tmp to being RAM based, as it gets used
> for large files sometimes, and .. oops. there goes your memory!...;-)

Or NOT as the case may be.

First, a tmpfs is mapped to memory in a way that slightly more efficient
than a disk based FS.  Yes, disk based FSs are mapped to memory,
buffers, for reading writing inodes and super-blocks as well as
shuffling the B-trees and indexes and more.  By comparison a tmps is
incredibly light weight.

Secondly, Linux uses a demand paged virtual memory  so you're never
going to run out of memory, for whatever value of 'never' applies.  And
it does apply here.  If that memory is needed by a process it can be
paged out to swap.

Thirdly, when it can, and that certainly applies for executables, Linux
and late model UNIX tries to "Map" a file into memory so the file is
actually demand paged - just like above.  Yes a programmer can open a
file so its not mapped, thinking he's smarter than the system designer
and knows better about that is and is not efficient, but I'd be
reluctant to hire such people as that reasoning would only apply in
special cases such as databases and the like.


I think you're worrying about something that isn't a concern.


Now if we're talking about my memory starved box from The Closet Of
Anxieties ... no, its still a NOT, because I'm not doing anything on
such a box that involves big files.  Its only a small box...




--
The early bird gets the worm, but the second mouse gets the cheese.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

Carlos E. R.-2
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



On Monday, 2013-05-06 at 21:50 -0400, Anton Aylward wrote:

> Or NOT as the case may be.
>
> First, a tmpfs is mapped to memory in a way that slightly more efficient
> than a disk based FS.  Yes, disk based FSs are mapped to memory,
> buffers, for reading writing inodes and super-blocks as well as
> shuffling the B-trees and indexes and more.  By comparison a tmps is
> incredibly light weight.
>
> Secondly, Linux uses a demand paged virtual memory  so you're never
> going to run out of memory, for whatever value of 'never' applies.  And
> it does apply here.  If that memory is needed by a process it can be
> paged out to swap.

If, for example, gimp needs to store a temporary file of 4 GB in /tmp, in
my computer that means it will swap (I have 8GiB ram, and swap is used
already). Swap being used that lot means that most of the system
applications will be impacted. On the other hand, if /tmp is disk, only
gimp is impacted.

Or... I read comments the other day of some one using k3b. It turned out
that dvd images were going to /tmp - again, huge usage.

Worse: many people nowdays do not even create swap! They think that
computers with 8 GiB are big enough.


No, thanks, I do not want /tmp in RAM.


- --
Cheers,
        Carlos E. R.
        (from 12.1 x86_64 "Asparagus" at Telcontar)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)

iEYEARECAAYFAlGIa/8ACgkQtTMYHG2NR9V1lACffuNoGKR5lyNjLRZR8Jf5NpIb
20sAnj4JyjYP4hk9n8WWkekCwp2fycYu
=rGFQ
-----END PGP SIGNATURE-----
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

Cristian Rodríguez-2
On 05/06/2013 10:50 PM, Carlos E. R. wrote:

> If, for example, gimp needs to store a temporary file of 4 GB in /tmp,
> in my computer that means it will swap (I have 8GiB ram, and swap is
> used already). Swap being used that lot means that most of the system
> applications will be impacted. On the other hand, if /tmp is disk, only
> gimp is impacted.
>
> Or... I read comments the other day of some one using k3b. It turned out
> that dvd images were going to /tmp - again, huge usage.
>
> Worse: many people nowdays do not even create swap! They think that
> computers with 8 GiB are big enough.

Both of those cases are bugs in the applications, they must create those
files in /var/tmp instead.

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

Carlos E. R.-3
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



On Monday, 2013-05-06 at 23:11 -0400, Cristian Rodríguez wrote:

> On 05/06/2013 10:50 PM, Carlos E. R. wrote:
>
>> If, for example, gimp needs to store a temporary file of 4 GB in /tmp,
>> in my computer that means it will swap (I have 8GiB ram, and swap is
>> used already). Swap being used that lot means that most of the system
>> applications will be impacted. On the other hand, if /tmp is disk, only
>> gimp is impacted.
>>
>> Or... I read comments the other day of some one using k3b. It turned out
>> that dvd images were going to /tmp - again, huge usage.
>>
>> Worse: many people nowdays do not even create swap! They think that
>> computers with 8 GiB are big enough.
>
> Both of those cases are bugs in the applications, they must create those
> files in /var/tmp instead.
Why is it a bug?

Who mandates what goes where?

/tmp has existed for eons, and you can not change what other software
developers do. If you make the distribution change /tmp to RAM, you piss
users instead, who are innocent bystanders.

Get all those applications to correct their act first (if you can convince
them), then consider if /tmp can be changed to RAM. Not otherwise.

- --
Cheers,
        Carlos E. R.
        (from 12.1 x86_64 "Asparagus" at Telcontar)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)

iEYEARECAAYFAlGIdSEACgkQtTMYHG2NR9Vg9QCcDyg2Vrc5eYJp8JxsuCsIhXCR
dxIAn29ZvVkufLYfuf5GQVYKN1fCq0V9
=KwEN
-----END PGP SIGNATURE-----
Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

Andrei Borzenkov
In reply to this post by L A Walsh
On Tue, May 7, 2013 at 5:18 AM, Linda Walsh <[hidden email]> wrote:
> The "tmpfs" is mounted on both /run and /var/run  --
> so it is the same memory based file system mounted at both points.
>
> That's why you see duplicates in both places

Two tmpfs mounts are completely different and separate filesystems.
They have different content. Otherwise /run and /dev would be
identical too.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

suse-8
In reply to this post by Anton Aylward-2
From: Anton Aylward <[hidden email]>
To: [hidden email]
Subject: Re: [opensuse] Re: why /var/run and /run these two directories
have absolutely the same contents?
Date: Mon, 06 May 2013 21:50:37 -0400

Linda Walsh said the following on 05/06/2013 09:18 PM:
> (which is a darn
> good reason not to move /tmp to being RAM based, as it gets used
> for large files sometimes, and .. oops. there goes your memory!...;-)

Or NOT as the case may be.

First, a tmpfs is mapped to memory in a way that slightly more efficient
than a disk based FS.  Yes, disk based FSs are mapped to memory,
buffers, for reading writing inodes and super-blocks as well as
shuffling the B-trees and indexes and more.  By comparison a tmps is
incredibly light weight.

Secondly, Linux uses a demand paged virtual memory  so you're never
going to run out of memory, for whatever value of 'never' applies.  And
it does apply here.  If that memory is needed by a process it can be
paged out to swap.

-----Original Message-----

It makes me wonder if you ever used k3b or kiwi....
Or are you blessed with 32GB unused mem or more at home?

Swap?
Some systems don't use swap, as it is a perfect way of destroying ssd.

(or do you use your mem as swap device ;-)

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

L A Walsh
In reply to this post by Andrei Borzenkov
Andrey Borzenkov wrote:
> On Tue, May 7, 2013 at 5:18 AM, Linda Walsh <[hidden email]> wrote:
>> The "tmpfs" is mounted on both /run and /var/run  --
>> so it is the same memory based file system mounted at both points.
>>
>> That's why you see duplicates in both places
>
> Two tmpfs mounts are completely different and separate filesystems.
> They have different content. Otherwise /run and /dev would be
> identical too.
---
        Hmmm...  you are right about them not being two separate
mount calls of the same filesystem.  I see that they use "bind":


        I was wrong about the dual mount, they do use bind:

mount -n --bind /run /var/run
mount -n --bind /run/lock /var/lock

/dev is not tmpfs but of type 'devtmpfs' -- a tmpfs backed
device that is pre-populated with standard device ID's as
a safety measure.

You may have an earlier version of the OS that doesn't have
the binding - but on my system -- creating a file in one
creates it in the other automatically.




--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

L A Walsh
In reply to this post by Anton Aylward-2
Anton Aylward wrote:

> Linda Walsh said the following on 05/06/2013 09:18 PM:
>> (which is a darn
>> good reason not to move /tmp to being RAM based, as it gets used
>> for large files sometimes, and .. oops. there goes your memory!...;-)
>
> Or NOT as the case may be.
>
> First, a tmpfs is mapped to memory in a way that slightly more efficient
> than a disk based FS.  Yes, disk based FSs are mapped to memory,
> buffers, for reading writing inodes and super-blocks as well as
> shuffling the B-trees and indexes and more.  By comparison a tmps is
> incredibly light weight.
---
        Sorry -- its not.  a 1TB file still takes 1TB out of the backing
store whether it is on disk or memory.


>
> Secondly, Linux uses a demand paged virtual memory  so you're never
> going to run out of memory, for whatever value of 'never' applies.  And
> it does apply here.  If that memory is needed by a process it can be
> paged out to swap.
---
        As someone else pointed out, it's ridiculous to have much virtual
memory these days -- it's a waste of disk space -- the kernel only
uses  it, *significantly*,  in pathological situations just before
the OOM-killer is invoked.


>
> Thirdly, when it can, and that certainly applies for executables, Linux
> and late model UNIX tries to "Map" a file into memory so the file is
> actually demand paged - just like above.  Yes a programmer can open a
> file so its not mapped, thinking he's smarter than the system designer
> and knows better about that is and is not efficient, but I'd be
> reluctant to hire such people as that reasoning would only apply in
> special cases such as databases and the like.
----
        You would get what you deserve.

        If you try to copy a memory mapped file and someone deleted it
or truncates while you are copying it -- your copy process dies.

        Programmers usually are smarter than system designers about what
their program does and needs to do.  System designers will all admit,
that optimal behavior is decided by the application and its writer.

        your example of 'databases' is exactly the case
that you wouldn't open -- you'd use mapped.  They are usually
fixed in size and you don't want to rewrite them completely,
but update fixed sized records.  It's ideal for memory mapping.
Are you sure you didn't get up on the wrong world this morning and
maybe your double on this world is on your world where things are
backwards from ours?  ;-)

        Anyfile that can be changed by other processes and rewritten
is not one you want to open using memory mapping.  Can you imagine
the performance penalty if someone or a library tried to do a memory
move or 'garbage collection' on something that was mapped to disk?
Ouch!


>
> I think you're worrying about something that isn't a concern.
---
        It's a major concern.  If some idiot steals /tmp for their
own selfish purposes, it will kill many apps.

        /var/run is for long-lived tmp files -- /tmp is for
short lived tmp files...  If I am copying files from one system
to another and need a tmp dir -- I use the one for short-lived files.

        Now can you tell me how /tmp mapped to your swap will handle
a 1-2 or 2-3 TB file?   Do you really have that much swap allocated?


> Now if we're talking about my memory starved box from The Closet Of
> Anxieties ... no, its still a NOT, because I'm not doing anything on
> such a box that involves big files.  Its only a small box...
----
        I wouldn't call a 48G server memory starved, exactly, but
the case can always be made for more.  yet it tops out at 192G capacity.

That doesn't come close to holding files in the TB range.

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

Joachim Schrod
In reply to this post by Anton Aylward-2
Anton Aylward wrote:

>
> First, a tmpfs is mapped to memory in a way that slightly more efficient
> than a disk based FS.  Yes, disk based FSs are mapped to memory,
> buffers, for reading writing inodes and super-blocks as well as
> shuffling the B-trees and indexes and more.  By comparison a tmps is
> incredibly light weight.
>
> Secondly, Linux uses a demand paged virtual memory  so you're never
> going to run out of memory, for whatever value of 'never' applies.  And
> it does apply here.  If that memory is needed by a process it can be
> paged out to swap.
>
>
> Now if we're talking about my memory starved box from The Closet Of
> Anxieties ... no, its still a NOT, because I'm not doing anything on
> such a box that involves big files.

I beg to differ. My current workstation (a Dell Optiplex) has 8 GB;
when I bought it two years ago that seemed to be sufficient. Start
two or three VM instances, two Eclipses, have Chrome and Firefox
running, and some associated programs, and you want to have the
rest of those 8 GB available for filesystem buffering -- and not
just for /tmp.

Add the tendency of programs to place very large files (several
GBs) in /tmp, and then you know why I think it's preferable to have
/tmp on disk. Btw, I want to use swapping only in dire
circumstances and not on a regular base; it affects all running
applications and not just those that have content on /tmp.

That's also the reason why I think that any file left over in /tmp
by an application after its termination is a bug in that
application. Applications should clean up after them and should
leave temporary files only after crashing. And then they *should*
leave them so that one can analyze the situation better...

Just my 0.02 €

        Joachim

--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Joachim Schrod, Roedermark, Germany
Email: [hidden email]

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

Anton Aylward-2
In reply to this post by Carlos E. R.-2
Carlos E. R. said the following on 05/06/2013 10:50 PM:

>
>
> On Monday, 2013-05-06 at 21:50 -0400, Anton Aylward wrote:
>
>> Or NOT as the case may be.
>
>> First, a tmpfs is mapped to memory in a way that slightly more efficient
>> than a disk based FS.  Yes, disk based FSs are mapped to memory,
>> buffers, for reading writing inodes and super-blocks as well as
>> shuffling the B-trees and indexes and more.  By comparison a tmps is
>> incredibly light weight.
>
>> Secondly, Linux uses a demand paged virtual memory  so you're never
>> going to run out of memory, for whatever value of 'never' applies.  And
>> it does apply here.  If that memory is needed by a process it can be
>> paged out to swap.
>
> If, for example, gimp needs to store a temporary file of 4 GB in /tmp, in
> my computer that means it will swap (I have 8GiB ram, and swap is used
> already). Swap being used that lot means that most of the system
> applications will be impacted. On the other hand, if /tmp is disk, only
> gimp is impacted.
>
> Or... I read comments the other day of some one using k3b. It turned out
> that dvd images were going to /tmp - again, huge usage.
>
> Worse: many people nowdays do not even create swap! They think that
> computers with 8 GiB are big enough.
>
>
> No, thanks, I do not want /tmp in RAM.

I did say "as the case may be".

I did not address performance.  You chose to.

"There's more than one way to do it".

There are many other cases where the performance boost of a tempfile in
memory is a Good Thing.  Some programs still hold over from the days
before virtual memory and they could not keep their 'arrays' in memory
so "overflowed' to disk.  Yes, they need to be redesigned so that the
arrays are in memory and they rely on the system paging in and out (to
swap) as needed rather than explicitly swapping in and out to a
temporary file.

YMMV.

Lets face it, for any context you choose to define you can find a case
where it is optimal for one thing and pessimal for another.  In general,
optimizing for any one thing has a cost elsewhere and a general purpose
compromise that is "good enough" for most things is also going to be
sub-optimal for all those same things.

If you want a system that's optimized for burning DVDs, that's fine.
If you want to use the system for detailed photo editing, that's fine.
You can optimise for each.  The flexibility and ease of such of Linux is
one of its great strengths.

But please don't bitch that that when you have a general purpose config
that it isn't optimal for everything you could throw at it.
--
"What we have learned from others becomes our own through reflection".
   - Ralph Emerson.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

Anton Aylward-2
In reply to this post by suse-8
Hans Witvliet said the following on 05/07/2013 03:06 AM:
> It makes me wonder if you ever used k3b or kiwi....

Please see my reply to Carlos.

And yes, I use k3b a lot: to copy CDs, to burn .iso files that I've
downloaded.

Your point being?

--
Hell must be like a karaoke bar
   - [hidden email]
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

Anton Aylward-2
In reply to this post by L A Walsh
Linda Walsh said the following on 05/07/2013 04:10 AM:
> Anton Aylward wrote:
>> Linda Walsh said the following on 05/06/2013 09:18 PM:
>>> (which is a darn
>>> good reason not to move /tmp to being RAM based, as it gets used
>>> for large files sometimes, and .. oops. there goes your memory!...;-)
>>
>> Or NOT as the case may be.

> Sorry -- its not.  a 1TB file still takes 1TB out of the backing
> store whether it is on disk or memory.

Ah, another fringe case raises its head!

To my mind a 1TB temp file tells me something is wrong, perhaps
something has run away and is producing prodigious output..
Carlos mentions overloading shared resources as being unsociable
(perhaps that's what cgroups are for but that's another matter -
obviously you're not using cgroups to limit that runaway process that is
generating the 1TB temp file) but I would think that consuming all of
/tmp with a runaway -- or is it just poorly designed -- application is
unsociable as well.

Or are you being ridiculous just for the sake of the argument?





>
>>
>> Secondly, Linux uses a demand paged virtual memory  so you're never
>> going to run out of memory, for whatever value of 'never' applies.  And
>> it does apply here.  If that memory is needed by a process it can be
>> paged out to swap.
> ---
> As someone else pointed out, it's ridiculous to have much virtual
> memory these days -- it's a waste of disk space -- the kernel only
> uses  it, *significantly*,  in pathological situations just before
> the OOM-killer is invoked.


I think you have the argument backward.

First, swap is there for the cases where VM does overflow.

Yes, if you have a well constrained system that isn't doing things which
demand memory, and you aren't running so many processes that some have
to swap out so others can run, they you can in fact do without swap or
end up not using swap.

Good luck to you.
My little 1G box from the Closet of Anxieties isn't in that fortunate
situation :-(

So if you have a sort program or similar that is chundling over a large
data set, backwards and forward, and needs working tables, with it needs
lots of memory or, it uses a temp file as intermediate store.  If the
latter, its nice if that intermediate store is fast.  Unless you're
using cgroups to limit things then this is another fringe case.  We can
always come up with fringe cases.  I can optimise for this particular
fringe case by giving it more memory; perhaps 'in core' if the design
uses in core tables and sort buffers, perhaps as shm if its an old
design that uses external files.  Either I have enough real memory to
accommodate it either way or I don't.  If I don't have enough real
memory then it doesn't matter whether it is running in core or with a
tmpfs temp file - its going to use VM and something is going to hit swap.

Any more fringe cases?

Perhaps we can optimise by spending money on a  high end graphics card
rather than memory?

Oh right, that's a fringe case as well.



>
> System designers will all admit,
> that optimal behavior is decided by the application and its writer.

I'm glad you go to that, Linda.
I'm glad you're saying that "optimal" depends on the "application".




> Anyfile that can be changed by other processes and rewritten
> is not one you want to open using memory mapping.  Can you imagine
> the performance penalty if someone or a library tried to do a memory
> move or 'garbage collection' on something that was mapped to disk?
> Ouch!

Yes, I'm glad you noticed that I mentioned SPECIFICALLY that mapping was
done with executables and libraries.

Oh, and yes, we do write to such files quite often.
Perhaps you'd care to run a 'zypper ps' after doing a 'zypper up' to see
an example of your "ouch".


>> I think you're worrying about something that isn't a concern.
> ---
> It's a major concern.  If some idiot steals /tmp for their
> own selfish purposes, it will kill many apps.

Oh, you mean like creating the 1TB file in /tmp you mentioned above.
Yes that would be idiotic, and if you're using a shared system its
idiotic not to use something like cgroups of other resource management
mechanism to let someone steal that much from /tmp.

Because in that and similar cases it really doesn't matter whether /tmp
is a disk or a tmpfs.  In that case worrying about that chose isn't a
concern, like I said.  Worrying about idiot or abusing users and
resource exhaustion in a shared environment is your concern.  Heck,
maybe that idiot is creating the 1TB file in /home/idiot/Documents/
rather than /tmp, and worry about the disk/tmpfs choice for /tmp isn't a
concern but resource exhaustion still is.







> /var/run is for long-lived tmp files -- /tmp is for
> short lived tmp files...  If I am copying files from one system
> to another and need a tmp dir -- I use the one for short-lived files.


Right.  So /tmp is for the short lived intermediate and buffer files for
the sort program or the phases of the compiler; and in those fringe
cases I can make the case that a tmpfs implementation is quite sensible;
get the program to run faster so it completes and removes them faster.


When I copy files from one system to another I use rsync - that's
another thread, though.  Does it use a temp file?




> Now can you tell me how /tmp mapped to your swap will handle
> a 1-2 or 2-3 TB file?   Do you really have that much swap allocated?

Are we talking my dinky little 1G memory system running on a 200G drive
from the Closet of Anxieties?

Hopefully you have a system that is geared for the kind of applications
you choose to run on it.

How many people have a /tmp file on disk that that can handle your 2-3
TB file?   That's the question.

Because if you're in the context where such files are the norm then
you're going to design a system around that use-case.


>> Now if we're talking about my memory starved box from The Closet Of
>> Anxieties ... no, its still a NOT, because I'm not doing anything on
>> such a box that involves big files.  Its only a small box...
> ----
> I wouldn't call a 48G server memory starved, exactly, but
> the case can always be made for more.  yet it tops out at 192G capacity.
>
> That doesn't come close to holding files in the TB range.

I think you're making my point for me, Linda.

Recall, my first line was

        Or NOT as the case may be

All the arguments against tmpfs has been specific use cases.
If you're not designing and administering your system to address the
business use cases then you re failing.  Arguing about disk vs tmpfs is
so missing the point.

You said above

> System designers will all admit,
> that optimal behavior is decided by the application and its writer.

The application is there to meet the business use case.
If that necessitates you finding a way to accommodate 2-3 TB temporary,
transient files AT AN ACCEPTABLE PERFORMANCE than so be it.  If Linux
can't hack it then call IBM and ask if they have a machine that can cope
with the use case.



There's an old Zen teaching about the finger pointing at the moon ...
The Enlightened look where the finger is pointing; the novice focuses on
the finger.





--
Sometimes I think your train of though is carrying a shipment of toxic
waste.
        -- Ozy & Millie, Monday, August 28, 2000
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

Anton Aylward-2
In reply to this post by Joachim Schrod
Joachim Schrod said the following on 05/07/2013 04:22 AM:

>
> I beg to differ.

Good, but what are you differing with?

My opening line, which people keep deleting was

        Or NOT as the case may be




> My current workstation (a Dell Optiplex) has 8 GB;
> when I bought it two years ago that seemed to be sufficient.

Sufficient for what?  Did you have a use-case in mind or was it just
"That was what available for what I was willing to spend"?
Its that "seemed" that bothers me.
How did you measure acceptability?

> Start
> two or three VM instances, two Eclipses, have Chrome and Firefox
> running, and some associated programs, and you want to have the
> rest of those 8 GB available for filesystem buffering -- and not
> just for /tmp.

We keep saying "YMMV".
My experience is that there are many cases where context switching kills
performance and that can be tuned.  I've also seen a lower speed,
multi-core system do better in the kind of workstation, interactive
setting than a single core system with the same memory, even before tuning.

There's also "what's active?"
I don't do code development, but I d a lot of writing and presentation
work.  Browser and file system and PDF readers, but I can shit down many
other things.  If I'm not going to use it for a while then its taking up
memory anyway!  The reality is that you're focusing on one thing at a
time.  If you system is fast enough then it *could* page everything else
out and all of the one thing that matters in.

In reality though you should tune your paging system as to how
aggressive it is about paging out unused parts so that the working set
of the apps you are switching between are resident.  So long as they are
resident little else matters.

What are they?  Well that depends, as I keep saying on you use case, and
whether tmpfs will help depends on the case - like I said above, the use
case.


> Add the tendency of programs to place very large files (several
> GBs) in /tmp, and then you know why I think it's preferable to have
> /tmp on disk.

Ah, not always.  Sometimes you want the content of that file back fast;
perhaps its a sort of lookup table.

It all depends on the use case.




> Btw, I want to use swapping only in dire
> circumstances and not on a regular base; it affects all running
> applications and not just those that have content on /tmp.

"Dire"?
I'm using the term paging here, not swapping.
"Swapping" implies a completeness, like in the PDP-11 days where the
whole program was swapped in and out in its completeness.

Hopefully your VM system isn't doing that.  Its a DEMAND PAGED virtual
memory system.  What gets paged out is paged out because (a) it hasn't
been used in a long time and (b) something else need the memory.

If you system is swapping out whole programs in their completeness so
that something else can run then yes your are in a dire situation.
Something is seriously wrong and the issue of disk vs tmpfs isn't the
cause and isn't a factor.

Paging, on the other hand, is the system's response to what you are
doing requiring more memory of the working set than is physically
available.  Its perfectly normal.  A well configured system will be
reluctant to page.  But when it does it does so because there are no
alternatives.

Yes you can tune the virtual memory in various ways.  Yes you'll need to
learn in order not to make a mess of it, but that applies to everything
in life from relations ships with your partner, though driving a car to
programming.


Methinks you're trying to assert some absolutes; you shouldn't.
There's a lot of YMMV with Linux.  Its why so many things, the Virtual
memory system being just one of them, is tunable.



> That's also the reason why I think that any file left over in /tmp
> by an application after its termination is a bug in that
> application. Applications should clean up after them and should
> leave temporary files only after crashing. And then they *should*
> leave them so that one can analyze the situation better...

I'd emphasise all the SHOULD in that paragraph and insert a few more.
But the reality is that we have to live with what we have.  Some
developers are reluctant to modify their code.  (Others are dead.)

--
"In the confrontation between the stream and the rock, the stream always
wins ? not through strength but by perseverance."
   -- H. Jackson Brown.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

Anton Aylward-2
In reply to this post by suse-8
Hans Witvliet said the following on 05/07/2013 03:06 AM:
> From: Anton Aylward <[hidden email]>

>
> -----Original Message-----
>

Please don't reply to BOTH the list AND me - its not necessary, I do
read the list.

Isn't there something about that in the guidelines?



--
One trend that bothers me is the glorification of stupidity, that the
media is reassuring people it's all right not to know anything. That
to me is far more dangerous than a little pornography on the Internet.
- Carl Sagan
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: why /var/run and /run these two directories have absolutely the same contents?

Carlos E. R.-2
In reply to this post by Anton Aylward-2
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



On Tuesday, 2013-05-07 at 07:51 -0400, Anton Aylward wrote:

> Carlos E. R. said the following on 05/06/2013 10:50 PM:

>> No, thanks, I do not want /tmp in RAM.


> YMMV.
>
> Lets face it, for any context you choose to define you can find a case
> where it is optimal for one thing and pessimal for another.  In general,
> optimizing for any one thing has a cost elsewhere and a general purpose
> compromise that is "good enough" for most things is also going to be
> sub-optimal for all those same things.

That's right, and I want that choice. I want to decide myself if I want
/tmp in RAM or not. The current openSUSE default of "not" is good for me.


> But please don't bitch that that when you have a general purpose config
> that it isn't optimal for everything you could throw at it.

I'm not bitching. :-|

- --
Cheers,
        Carlos E. R.
        (from 12.1 x86_64 "Asparagus" at Telcontar)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)

iEYEARECAAYFAlGJLhQACgkQtTMYHG2NR9UUlQCbB3iwY24IkOwMd2oY+yPYVz0C
oEUAnRkg+LMvYY8uWGHjke3/gyi6ZYGm
=T32r
-----END PGP SIGNATURE-----
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

1234