fdisk calculations

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

fdisk calculations

jdd@dodin.org
Hello :-)

I'm revising the partition HOWTO, using openSUSE 11.1 as example.

I try to understand fdisk output. namely this one from an USB key:

..........................
Disk /dev/sdb: 2012 MB, 2012217344 bytes
47 heads, 46 sectors/track, 1817 cylinders
Units = cylinders of 2162 * 512 = 1106944 bytes
Disk identifier: 0x04030201

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1818     1964932    6  FAT16
.......................

I can't understand the underlying calculus. How is the byte number found?
47 heads * 46 s/t = 2162, ok
2162 * 512 = 1106944, ok
but
1817 cyl * 1106944 bytes = 2011317248, not 2012217344 as shown

any idea?

thanks
jdd

--
http://www.dodin.net
http://valerie.dodin.org
http://www.youtube.com/watch?v=t-eic8MSSfM
http://www.facebook.com/profile.php?id=1412160445

--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: fdisk calculations

Steffen Winterfeldt
On Mon, 16 Mar 2009, jdd wrote:

> Hello :-)
>
> I'm revising the partition HOWTO, using openSUSE 11.1 as example.
>
> I try to understand fdisk output. namely this one from an USB key:
>
> ..........................
> Disk /dev/sdb: 2012 MB, 2012217344 bytes
> 47 heads, 46 sectors/track, 1817 cylinders
> Units = cylinders of 2162 * 512 = 1106944 bytes
> Disk identifier: 0x04030201
>
>    Device Boot      Start         End      Blocks   Id  System
> /dev/sdb1               1        1818     1964932    6  FAT16
> .......................
>
> I can't understand the underlying calculus. How is the byte number found?
> 47 heads * 46 s/t = 2162, ok
> 2162 * 512 = 1106944, ok
> but
> 1817 cyl * 1106944 bytes = 2011317248, not 2012217344 as shown
>
> any idea?

2012217344/512/46/47 = 1817.813...

2012217344 is the disk size. And that's independent of the 'geometry'.


Steffen

--
Der frühe Wirt holt sich den Wurm.
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: fdisk calculations

jdd@dodin.org
In reply to this post by jdd@dodin.org
Steffen Winterfeldt a écrit :

> 2012217344/512/46/47 = 1817.813...
>
> 2012217344 is the disk size. And that's independent of the 'geometry'.

why should it be? the geometry size is the only available and OS
independant.

In fact this number gives 1919 mebibytes (1919*2^20=2012217344) isn't
1919 prime?

I wonder how all this is built. Where are used the 900.096 bytes left?

is the 2012217344 including extra bytes in sectors for sync/alignment
and whatsoever the drive need? but we are here on flash drive, not
magnetic, so what?

thanks
jdd

--
http://www.dodin.net
http://valerie.dodin.org
http://www.youtube.com/watch?v=t-eic8MSSfM
http://www.facebook.com/profile.php?id=1412160445

--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: fdisk calculations

Carlos E. R.-2
In reply to this post by jdd@dodin.org
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



On Monday, 2009-03-16 at 09:11 +0100, jdd wrote:

> Hello :-)
>
> I'm revising the partition HOWTO, using openSUSE 11.1 as example.
>
> I try to understand fdisk output. namely this one from an USB key:
>
> ..........................
> Disk /dev/sdb: 2012 MB, 2012217344 bytes
> 47 heads, 46 sectors/track, 1817 cylinders
> Units = cylinders of 2162 * 512 = 1106944 bytes
> Disk identifier: 0x04030201
>
>   Device Boot      Start         End      Blocks   Id  System
> /dev/sdb1               1        1818     1964932    6  FAT16
> .......................
>
> I can't understand the underlying calculus. How is the byte number found?
> 47 heads * 46 s/t = 2162, ok
> 2162 * 512 = 1106944, ok
> but
> 1817 cyl * 1106944 bytes = 2011317248, not 2012217344 as shown
>
> any idea?

Maybe the real size is 2012217344, ie, 1919 MiB. The head/cyls/sector
count is an aproximation for the shake of old bios functions. New disk use
LBA addressing, ie, sector counts. You can use "u" to change the units in
fdisk. Also, you can use "v" to do a verification, which usually gives a
number of non allocated sectors; I'm not sure, but could be related to
that mismatch.

One of mine:

Disk /dev/hdb: 40.0 GB, 40020664320 bytes
255 heads, 63 sectors/track, 4865 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x7bdf8b0e

or:

Disk /dev/hdb: 40.0 GB, 40020664320 bytes
255 heads, 63 sectors/track, 4865 cylinders, total 78165360 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x7bdf8b0e

Command (m for help): v
9321 unallocated sectors

But:

255*63*4865 = 78156225 sectors

78165360 - 78156225 = 9135

- --
Cheers,
        Carlos E. R.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkm+R70ACgkQtTMYHG2NR9X82wCaA1yB1CQpbbQOF03k2YRzzJLN
mZQAn1IC6lyIs9dqsx64flodVvhbeHns
=dQxR
-----END PGP SIGNATURE-----
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Re: fdisk calculations

Randall Schulz
In reply to this post by jdd@dodin.org
On Monday March 16 2009, jdd wrote:
> ...
>
> In fact this number gives 1919 mebibytes (1919*2^20=2012217344) isn't
> 1919 prime?

Speculation is not required (nor pesky arithmetic):

% factor 1919
1919: 19 101


> ...
>
> thanks
> jdd


Randall Schulz
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Re: fdisk calculations

Steffen Winterfeldt
In reply to this post by jdd@dodin.org
On Mon, 16 Mar 2009, jdd wrote:

> Steffen Winterfeldt a écrit :
>
> > 2012217344/512/46/47 = 1817.813...
> >
> > 2012217344 is the disk size. And that's independent of the 'geometry'.
>
> why should it be? the geometry size is the only available and OS
> independant.

Geometry values are just gimmick and not really used for anything but for
confusing messages.

> In fact this number gives 1919 mebibytes (1919*2^20=2012217344) isn't
> 1919 prime?
>
> I wonder how all this is built. Where are used the 900.096 bytes left?

As has already been posted, 1919 is 101*19; so if you are worried about that
disk space, change the geometry to something like 101 heads x 19
sectors/track.


Steffen

--
Der frühe Wirt holt sich den Wurm.
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: fdisk calculations

jdd@dodin.org
Steffen Winterfeldt a écrit :

> As has already been posted, 1919 is 101*19; so if you are worried about that
> disk space, change the geometry to something like 101 heads x 19
> sectors/track.

not really worried, only curious. flash memories are made from
discrete chips, so shouldn't the value be some kind of exact bit count?

anyway I aksed on the fdisk list - the first answer speaks about bug,
I will follow

jdd

--
http://www.dodin.net
http://valerie.dodin.org
http://www.youtube.com/watch?v=t-eic8MSSfM
http://www.facebook.com/profile.php?id=1412160445

--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: fdisk calculations

gregfreemyer
In reply to this post by jdd@dodin.org
On Mon, Mar 16, 2009 at 4:11 AM, jdd <[hidden email]> wrote:

> Hello :-)
>
> I'm revising the partition HOWTO, using openSUSE 11.1 as example.
>
> I try to understand fdisk output. namely this one from an USB key:
>
> ..........................
> Disk /dev/sdb: 2012 MB, 2012217344 bytes
> 47 heads, 46 sectors/track, 1817 cylinders
> Units = cylinders of 2162 * 512 = 1106944 bytes
> Disk identifier: 0x04030201
>
>   Device Boot      Start         End      Blocks   Id  System
> /dev/sdb1               1        1818     1964932    6  FAT16
> .......................
>
> I can't understand the underlying calculus. How is the byte number found?
> 47 heads * 46 s/t = 2162, ok
> 2162 * 512 = 1106944, ok
> but
> 1817 cyl * 1106944 bytes = 2011317248, not 2012217344 as shown
>
> any idea?
>
> thanks
> jdd

jdd,

I would simply have th how-to say something like:

==
The Cylinder / Head / Sector concept is an anachronism from the 1980's
and early 1990's.  The CHS concept can only describe disks with a max
capacity of approximately 4 GB.  (ie. (2^16-1)*(2^8 -1)*(2^5-1)*512
bytes).

(You need to verify the number of bits available per field, I'm
working from memory.)

Thus the vast majority of currently shipping disks simply use the
maximum allowed value for the number of cylinders, heads, and sectors
in an effort to allow low-level legacy code to access as much of the
disk drive as possible.

Boot loaders were some of the last generally used programs to only
support disk access via CHS.  For this reason even into the early 21st
century the boot partition had to be in this early portion of the
disk.

One specific legacy remnant of this design you may need to be
concerned with is that during the 1980s and 1990's, the first
partition was typically located at cylinder 0, head 1, sector 0 to
ensure the partition alignment was consistent with the physical disk.
With most disk drives today, that is sector 63.  This location for the
start of the first partition continues to be a common starting point,
but when Microsoft introduced Vista they started placing the first
partition at the 1MiB point.  Thus sector 2048 is now a common
starting sector for the first partition.

This has caused a conundrum that is being faced by hard drive
manufacturers in the early 2009 timeframe.  They are working on disk
drives that continue to support a 512 byte sector size on the
interface side, but use 4K sectors on the actual platter.  This means
that all physical reads and writes to the platters will take place
with 4K of data at a time.

This 4K size works well with most modern OS'es because they use a 4K
page size internally (or a multiple thereof).  The issue is alignment,
when a page is written by the OS to the hard drive it will be
important from a performance perspective that the page not overlap two
separate physical 4K sectors.

Unfortunately hard drive manufacturers have two common, but
contradictory alignment needs.  They can either have the 4K physical
sectors be aligned with sector 63 that has historically been used for
the start of the first partition, or they can be aligned with
partitions that start on sector 2048 that Vista uses.

At present it appears that manufacturers are leaning toward having the
hard drives configured in the factory to have one alignment or the
other.  And the user will have no ability to change it.

Thus if you are attempting to partition a new generation hard drive
that has 4K physical sectors, you should pay special attention to
where your partitions start.  The specific desire is that all
partitions be aligned with the 4K physical sectors of the drive. To
the best of my knowledge, as of Mar. 2009 none of the Linux Userspace
partitioning tools have been updated to address this issue because
shipping disk drives are not yet available to test with.
===

Hope that makes sense
Greg
--
Greg Freemyer
Head of EDD Tape Extraction and Processing team
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf

The Norcross Group
The Intersection of Evidence & Technology
http://www.norcrossgroup.com
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Re: fdisk calculations

Herbert Graeber-2
In reply to this post by Steffen Winterfeldt
Steffen Winterfeldt schrieb:

> On Mon, 16 Mar 2009, jdd wrote:
>
>  
>> Steffen Winterfeldt a écrit :
>>
>>    
>>> 2012217344/512/46/47 = 1817.813...
>>>
>>> 2012217344 is the disk size. And that's independent of the 'geometry'.
>>>      
>> why should it be? the geometry size is the only available and OS
>> independant.
>>    
>
> Geometry values are just gimmick and not really used for anything but for
> confusing messages.
>  
No, especially for SSDs careful selection of Geometry values can be
important:

   
http://thunk.org/tytso/blog/2009/02/20/aligning-filesystems-to-an-ssds-erase-block-size/

According to the nr. of heads* nr. sectors/head should a multiple of the
erase-block size (most often 128MB), and all partitions should be
aligned to cylinder boundaries. Especially the first partition of a disk
is usually not aligned by default, because the first track, not
cylinder, is excluded from it when you specify 1 as the starting cylinder.

Not respecting the erase block size leads to file system block split
over two erase blocks and therefore to worse performance and more erase
blocks erased than necessary and so to shorter life time of the device.

> [...]

Herbert
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Re: fdisk calculations

gregfreemyer
On Mon, Mar 16, 2009 at 10:30 AM, Herbert Graeber <[hidden email]> wrote:

> Steffen Winterfeldt schrieb:
>> On Mon, 16 Mar 2009, jdd wrote:
>>
>>
>>> Steffen Winterfeldt a écrit :
>>>
>>>
>>>> 2012217344/512/46/47 = 1817.813...
>>>>
>>>> 2012217344 is the disk size. And that's independent of the 'geometry'.
>>>>
>>> why should it be? the geometry size is the only available and OS
>>> independant.
>>>
>>
>> Geometry values are just gimmick and not really used for anything but for
>> confusing messages.
>>
> No, especially for SSDs careful selection of Geometry values can be
> important:
>
>
> http://thunk.org/tytso/blog/2009/02/20/aligning-filesystems-to-an-ssds-erase-block-size/
>
> According to the nr. of heads* nr. sectors/head should a multiple of the
> erase-block size (most often 128MB), and all partitions should be
> aligned to cylinder boundaries. Especially the first partition of a disk
> is usually not aligned by default, because the first track, not
> cylinder, is excluded from it when you specify 1 as the starting cylinder.
>
> Not respecting the erase block size leads to file system block split
> over two erase blocks and therefore to worse performance and more erase
> blocks erased than necessary and so to shorter life time of the device.
>
>> [...]
>
> Herbert

Herbert,

I agree with you for the first partition.  After that I don't think
tools like fdisk attempt to align with cylinder boundaries anymore.

Given your comment above and the email I just posted 5 minutes ago
about the new 4K alignment issues we will be seeing within the year,
it sounds like it is time for partitioning tools to totally drop the
concept of aligning with cylinders and start asking questions more
relevant to the hardware.  Specifically "what is the physical sector
size (unit of atomicity) for the device?  And what is the offset into
the the device represented by the very beginning of the drive.

Greg



--
Greg Freemyer
Head of EDD Tape Extraction and Processing team
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf

The Norcross Group
The Intersection of Evidence & Technology
http://www.norcrossgroup.com
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: fdisk calculations

jdd@dodin.org
In reply to this post by Herbert Graeber-2
Greg Freemyer a écrit :

> Given your comment above and the email I just posted 5 minutes ago
> about the new 4K alignment issues we will be seeing within the year,

all this is very interesting, thanks.

I use fdisk mostly because it's a "do all" thing. As far as I remember
the usb key partitionning is the factory one (I don't remember
changing this one) and not any partitionner accept to load such device.

I evne happen frequently to use loop mounted files to experiment and
only fdisk can work with these files.

jdd


--
http://www.dodin.net
http://valerie.dodin.org
http://www.youtube.com/watch?v=t-eic8MSSfM
http://www.facebook.com/profile.php?id=1412160445

--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Re: fdisk calculations

Herbert Graeber-2
In reply to this post by gregfreemyer
Greg Freemyer schrieb:

> On Mon, Mar 16, 2009 at 10:30 AM, Herbert Graeber <[hidden email]> wrote:
>  
>> Steffen Winterfeldt schrieb:
>>    
>>> On Mon, 16 Mar 2009, jdd wrote:
>>>
>>>
>>>      
>>>> Steffen Winterfeldt a écrit :
>>>>
>>>>
>>>>        
>>>>> 2012217344/512/46/47 = 1817.813...
>>>>>
>>>>> 2012217344 is the disk size. And that's independent of the 'geometry'.
>>>>>
>>>>>          
>>>> why should it be? the geometry size is the only available and OS
>>>> independant.
>>>>
>>>>        
>>> Geometry values are just gimmick and not really used for anything but for
>>> confusing messages.
>>>
>>>      
>> No, especially for SSDs careful selection of Geometry values can be
>> important:
>>
>>
>> http://thunk.org/tytso/blog/2009/02/20/aligning-filesystems-to-an-ssds-erase-block-size/
>>
>> According to the nr. of heads* nr. sectors/head should a multiple of the
>> erase-block size (most often 128MB), and all partitions should be
>> aligned to cylinder boundaries. Especially the first partition of a disk
>> is usually not aligned by default, because the first track, not
>> cylinder, is excluded from it when you specify 1 as the starting cylinder.
>>
>> Not respecting the erase block size leads to file system block split
>> over two erase blocks and therefore to worse performance and more erase
>> blocks erased than necessary and so to shorter life time of the device.
>>
>>    
>>> [...]
>>>      
> I agree with you for the first partition.  After that I don't think
> tools like fdisk attempt to align with cylinder boundaries anymore.
>  
fdisk and other partitioning tools can work without aligning to
cylinders. But when creating new partitions they usually respect
cylinder boundaries. When you need special block sizes, eg. for SSDs or
partitions on raid devices, you can use the -H and -S options to
override the values 63 for sectors/track and 255 for heads.
> Given your comment above and the email I just posted 5 minutes ago
> about the new 4K alignment issues we will be seeing within the year,
> it sounds like it is time for partitioning tools to totally drop the
> concept of aligning with cylinders and start asking questions more
> relevant to the hardware.  Specifically "what is the physical sector
> size (unit of atomicity) for the device?  And what is the offset into
> the the device represented by the very beginning of the drive
It is not possible to drop the geometry completely, because this build
deeply into the BIOS and the kernel.

Theoretically it would be possible to query the device for it's block
size and select a proper geometry for partitioning. But this does not
work, because many devices lie about their real geometry to be
compatible to older Windows OSs.

Herbert
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Re: fdisk calculations

gregfreemyer
On Mon, Mar 16, 2009 at 11:20 AM, Herbert Graeber <[hidden email]> wrote:

> Greg Freemyer schrieb:
>> On Mon, Mar 16, 2009 at 10:30 AM, Herbert Graeber <[hidden email]> wrote:
>>
>>> Steffen Winterfeldt schrieb:
>>>
>>>> On Mon, 16 Mar 2009, jdd wrote:
>>>>
>>>>
>>>>
>>>>> Steffen Winterfeldt a écrit :
>>>>>
>>>>>
>>>>>
>>>>>> 2012217344/512/46/47 = 1817.813...
>>>>>>
>>>>>> 2012217344 is the disk size. And that's independent of the 'geometry'.
>>>>>>
>>>>>>
>>>>> why should it be? the geometry size is the only available and OS
>>>>> independant.
>>>>>
>>>>>
>>>> Geometry values are just gimmick and not really used for anything but for
>>>> confusing messages.
>>>>
>>>>
>>> No, especially for SSDs careful selection of Geometry values can be
>>> important:
>>>
>>>
>>> http://thunk.org/tytso/blog/2009/02/20/aligning-filesystems-to-an-ssds-erase-block-size/
>>>
>>> According to the nr. of heads* nr. sectors/head should a multiple of the
>>> erase-block size (most often 128MB), and all partitions should be
>>> aligned to cylinder boundaries. Especially the first partition of a disk
>>> is usually not aligned by default, because the first track, not
>>> cylinder, is excluded from it when you specify 1 as the starting cylinder.
>>>
>>> Not respecting the erase block size leads to file system block split
>>> over two erase blocks and therefore to worse performance and more erase
>>> blocks erased than necessary and so to shorter life time of the device.
>>>
>>>
>>>> [...]
>>>>
>> I agree with you for the first partition.  After that I don't think
>> tools like fdisk attempt to align with cylinder boundaries anymore.
>>
> fdisk and other partitioning tools can work without aligning to
> cylinders. But when creating new partitions they usually respect
> cylinder boundaries. When you need special block sizes, eg. for SSDs or
> partitions on raid devices, you can use the -H and -S options to
> override the values 63 for sectors/track and 255 for heads.
>> Given your comment above and the email I just posted 5 minutes ago
>> about the new 4K alignment issues we will be seeing within the year,
>> it sounds like it is time for partitioning tools to totally drop the
>> concept of aligning with cylinders and start asking questions more
>> relevant to the hardware.  Specifically "what is the physical sector
>> size (unit of atomicity) for the device?  And what is the offset into
>> the the device represented by the very beginning of the drive
> It is not possible to drop the geometry completely, because this build
> deeply into the BIOS and the kernel.
>
> Theoretically it would be possible to query the device for it's block
> size and select a proper geometry for partitioning. But this does not
> work, because many devices lie about their real geometry to be
> compatible to older Windows OSs.
>
> Herbert

At a minimum tools like fdisk need to start asking the user about
these parameters.  The best partition layouts simply don't correlate
to CHS values anymore.

I don't know what the solution is when the physical device is claiming
a sectors per track value that is not a multiple of its true atomic
read/write region.  ie. like the 4k sector hdd manufacturers are
currently planning.

From a performance perspective alignment with physical atomic write
areas is critical, put I can see where that could break some boot
code.  It is very convoluted.  I just posted to the lkml-ide list that
they should try to encourage hardware manufacturers of the new 4K
sector hard drives to report 56 sectors per track.  I think that would
drastically simplify things.

Greg
--
Greg Freemyer
Head of EDD Tape Extraction and Processing team
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf

The Norcross Group
The Intersection of Evidence & Technology
http://www.norcrossgroup.com
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: fdisk calculations

David Haller-3
In reply to this post by gregfreemyer
Hello,

On Mon, 16 Mar 2009, Greg Freemyer wrote:

>On Mon, Mar 16, 2009 at 4:11 AM, jdd <[hidden email]> wrote:
>> I'm revising the partition HOWTO, using openSUSE 11.1 as example.
>>
>> I try to understand fdisk output. namely this one from an USB key:
>>
>> ..........................
>> Disk /dev/sdb: 2012 MB, 2012217344 bytes
>> 47 heads, 46 sectors/track, 1817 cylinders
>> Units = cylinders of 2162 * 512 = 1106944 bytes
>> Disk identifier: 0x04030201
>>
>>   Device Boot      Start         End      Blocks   Id  System
>> /dev/sdb1               1        1818     1964932    6  FAT16
>> .......................
>>
>> I can't understand the underlying calculus. How is the byte number found?
>> 47 heads * 46 s/t = 2162, ok
>> 2162 * 512 = 1106944, ok
>> but
>> 1817 cyl * 1106944 bytes = 2011317248, not 2012217344 as shown
>>
>> any idea?

Cylinders of 2162 sectors don't map to exactly 2012217344 Bytes.

$ echo 'sz=2012217344;cyl=(2162*512);c=sz/cyl;u=c*cyl;
   print c," Cyls = ", u, " Bytes\n",\
   sz-u, " Bytes ~= ", (sz-u)/512, " unused Sectors\n";\
   print 2012217344 - 2011317248, " Bytes\n";' | bc
1817 Cyls = 2011317248 Bytes
900096 Bytes ~= 1758 unused Sectors
900096 Bytes

You're missing exactly the bytes that don't match to the geometry.

You could use a different geometry to minimize the unused portion of
the disk. BTW: 47 heads / 46 sectors is a very weird geometry.

fdisk will tell you this when you "verify" the partitioning.

You could try the other "standard" geometry of * Cyls/16 Heads/63
Sectors. That leaves only 928 unused Sectors (~464 KB).

>I would simply have th how-to say something like:
>
>==
>The Cylinder / Head / Sector concept is an anachronism from the 1980's
>and early 1990's.  The CHS concept can only describe disks with a max
>capacity of approximately 4 GB.  (ie. (2^16-1)*(2^8 -1)*(2^5-1)*512
>bytes).

Correct is: ( 2^10 * 2 ^ 8 * 2 ^ 6 ) - 1 sectors. The 2 highest bits
of the sector-byte are used as the highest bits of the cylinder. Thus
yielding a max of 16777215 sectors = 8589934080 B = 8191 MB that are
theoretically adressable via CHS. IIRC though, the highest values
were/are used as a "this is a dummy value", thus it's only 1022 * 254
* 63 sectors ~= 7985 MB.

>(You need to verify the number of bits available per field, I'm
>working from memory.)

Once you've reconstructed an MBR from scratch, using pencil, paper, a
calculator for hex->bin->dec conversions and DOS debug.com, you won't
forget that fast ;)

>Thus the vast majority of currently shipping disks simply use the
>maximum allowed value for the number of cylinders, heads, and sectors
>in an effort to allow low-level legacy code to access as much of the
>disk drive as possible.

A C/H/S of 1023/255/63 marks the CHS-Entry as a "dummy"-Entry,
i.e. only the LBA should be used (which only addresses up to 127 GB,
as only 28 bits are used). The LBA-48 extension (from Maxtor,
specified since ATA-6 IIRC) fixes that for now, addressing up to 128
petabytes.

>Boot loaders were some of the last generally used programs to only
>support disk access via CHS.  For this reason even into the early 21st
>century the boot partition had to be in this early portion of the
>disk.

Or inside the first 127 GB, more recently, when LBA-48 support is
missing.

[..]
>Hope that makes sense

dito,
-dnh

PS@jdd: which howto? URL?

--
"Human beings make life so interesting. Do you know, that in a universe
 so full of wonders, they have managed to invent boredom."     -- Death
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: fdisk calculations

David Haller-3
In reply to this post by Carlos E. R.-2
Hello,

On Mon, 16 Mar 2009, Carlos E. R. wrote:

>Disk /dev/hdb: 40.0 GB, 40020664320 bytes
>255 heads, 63 sectors/track, 4865 cylinders, total 78165360 sectors
>Units = sectors of 1 * 512 = 512 bytes
>Disk identifier: 0x7bdf8b0e
>
>Command (m for help): v
>9321 unallocated sectors
>
>But:
>
>255*63*4865 = 78156225 sectors
>
>78165360 - 78156225 = 9135

There are 62 unallocated sectors after the MBR, and after each EPBR
(before each logical partition) -- but I don't know if fdisk counts
these as such. Show the complete 'fdisk -l' output (via PM) ...

-dnh

--
Troll, troll, troll along
gently down the feed.
Merrily, merrily, merrily
a life is what you need.               -- Stephan Lange in dang
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: fdisk calculations

jdd@dodin.org
In reply to this post by Carlos E. R.-2
David Haller a écrit :

> these as such. Show the complete 'fdisk -l' output (via PM) ...

shows exacltly the same thing I posted in my first post.

This discussion is very interesting, even if it go much more
complicated I thought initially.

I'm the "Partition Rescue" LDP HOWTO author and become recently the
LDP coordinator. I was sometime asked to include my HOWTO in the
Partition HOWTO and, as the Partition HOWTO author didn't want to
continue with it I decided to take the HOWTO authorship.

But the present HOWTO:
http://tldp.org/HOWTO/Partition/index.html
looks to me much too old, given all the new partitionning tools and
disk usage we have now, so I began rewriting all this. It's *not* yet
done. Drafts are here
http://wiki.tldp.org/Partitions-and-mass-storage-HOWTO

Some years ago, I made a course and for my students sake made use of
loop mounted files as partition and file system testbed. I may use
this also now for examples, but it's also easy to use usb keys, now so
cheap user can (eventually) destroy one without real problem :-)

But I don't want to write on a subject before understanding it fully.
Thas is why I did this post.

among others, a curious thing is that I perfectly remember that at the
moment LBA was introduced, it was presented as a way to show CHS with
fake numbers.

I have yet to control this, but ATM most Bioses showed different CHS
layout with LBA activated or not, but never shown direct sector
numbers. It's only when Linux come than this direct adressing was used
and never in partitionning tools.

When fdisk advertise the 1024 cylinder barrier, I don't have found PC
with this problem for more than 10 years now.

My goal is to try making clear the bare minimum necessary to make
actual partitionning on fixed and also external disks.

as of LBA, this seems pretty cool:
http://www.dewassoc.com/kbase/hard_drives/lba.htm

thanks
jdd

--
http://www.dodin.net
http://valerie.dodin.org
http://www.youtube.com/watch?v=t-eic8MSSfM
http://www.facebook.com/profile.php?id=1412160445

--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: fdisk calculations

Dave Plater
In reply to this post by David Haller-3
David Haller wrote:

> Hello,
>
> On Mon, 16 Mar 2009, Greg Freemyer wrote:
>  
>> On Mon, Mar 16, 2009 at 4:11 AM, jdd <[hidden email]> wrote:
>>    
>>> I'm revising the partition HOWTO, using openSUSE 11.1 as example.
>>>
>>> I try to understand fdisk output. namely this one from an USB key:
>>>
>>> ..........................
>>> Disk /dev/sdb: 2012 MB, 2012217344 bytes
>>> 47 heads, 46 sectors/track, 1817 cylinders
>>> Units = cylinders of 2162 * 512 = 1106944 bytes
>>> Disk identifier: 0x04030201
>>>
>>>   Device Boot      Start         End      Blocks   Id  System
>>> /dev/sdb1               1        1818     1964932    6  FAT16
>>> .......................
>>>
>>> I can't understand the underlying calculus. How is the byte number found?
>>> 47 heads * 46 s/t = 2162, ok
>>> 2162 * 512 = 1106944, ok
>>> but
>>> 1817 cyl * 1106944 bytes = 2011317248, not 2012217344 as shown
>>>
>>> any idea?
>>>      
>
> Cylinders of 2162 sectors don't map to exactly 2012217344 Bytes.
>
> $ echo 'sz=2012217344;cyl=(2162*512);c=sz/cyl;u=c*cyl;
>    print c," Cyls = ", u, " Bytes\n",\
>    sz-u, " Bytes ~= ", (sz-u)/512, " unused Sectors\n";\
>    print 2012217344 - 2011317248, " Bytes\n";' | bc
> 1817 Cyls = 2011317248 Bytes
> 900096 Bytes ~= 1758 unused Sectors
> 900096 Bytes
>
> You're missing exactly the bytes that don't match to the geometry.
>
> You could use a different geometry to minimize the unused portion of
> the disk. BTW: 47 heads / 46 sectors is a very weird geometry.
>
> fdisk will tell you this when you "verify" the partitioning.
>
> You could try the other "standard" geometry of * Cyls/16 Heads/63
> Sectors. That leaves only 928 unused Sectors (~464 KB).
>
>  
>> I would simply have th how-to say something like:
>>
>> ==
>> The Cylinder / Head / Sector concept is an anachronism from the 1980's
>> and early 1990's.  The CHS concept can only describe disks with a max
>> capacity of approximately 4 GB.  (ie. (2^16-1)*(2^8 -1)*(2^5-1)*512
>> bytes).
>>    
>
> Correct is: ( 2^10 * 2 ^ 8 * 2 ^ 6 ) - 1 sectors. The 2 highest bits
> of the sector-byte are used as the highest bits of the cylinder. Thus
> yielding a max of 16777215 sectors = 8589934080 B = 8191 MB that are
> theoretically adressable via CHS. IIRC though, the highest values
> were/are used as a "this is a dummy value", thus it's only 1022 * 254
> * 63 sectors ~= 7985 MB.
>
>  
>> (You need to verify the number of bits available per field, I'm
>> working from memory.)
>>    
>
> Once you've reconstructed an MBR from scratch, using pencil, paper, a
> calculator for hex->bin->dec conversions and DOS debug.com, you won't
> forget that fast ;)
>
>  
>> Thus the vast majority of currently shipping disks simply use the
>> maximum allowed value for the number of cylinders, heads, and sectors
>> in an effort to allow low-level legacy code to access as much of the
>> disk drive as possible.
>>    
>
> A C/H/S of 1023/255/63 marks the CHS-Entry as a "dummy"-Entry,
> i.e. only the LBA should be used (which only addresses up to 127 GB,
> as only 28 bits are used). The LBA-48 extension (from Maxtor,
> specified since ATA-6 IIRC) fixes that for now, addressing up to 128
> petabytes.
>
>  
>> Boot loaders were some of the last generally used programs to only
>> support disk access via CHS.  For this reason even into the early 21st
>> century the boot partition had to be in this early portion of the
>> disk.
>>    
>
> Or inside the first 127 GB, more recently, when LBA-48 support is
> missing.
>
> [..]
>  
>> Hope that makes sense
>>    
>
> dito,
> -dnh
>
> PS@jdd: which howto? URL?
>
>  
For what its worth windows 98se worked better with an 80G hard drive
when the chs values in the partition table were set to 0:0:0 and only
the sector values were true.
Regards
Dave P

--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Re: fdisk calculations

gregfreemyer
In reply to this post by jdd@dodin.org
On Tue, Mar 17, 2009 at 3:25 AM, jdd <[hidden email]> wrote:

> David Haller a écrit :
>
>> these as such. Show the complete 'fdisk -l' output (via PM) ...
>
> shows exacltly the same thing I posted in my first post.
>
> This discussion is very interesting, even if it go much more
> complicated I thought initially.
>
> I'm the "Partition Rescue" LDP HOWTO author and become recently the
> LDP coordinator. I was sometime asked to include my HOWTO in the
> Partition HOWTO and, as the Partition HOWTO author didn't want to
> continue with it I decided to take the HOWTO authorship.
>
> But the present HOWTO:
> http://tldp.org/HOWTO/Partition/index.html
> looks to me much too old, given all the new partitionning tools and
> disk usage we have now, so I began rewriting all this. It's *not* yet
> done. Drafts are here
> http://wiki.tldp.org/Partitions-and-mass-storage-HOWTO
>
> Some years ago, I made a course and for my students sake made use of
> loop mounted files as partition and file system testbed. I may use
> this also now for examples, but it's also easy to use usb keys, now so
> cheap user can (eventually) destroy one without real problem :-)
>
> But I don't want to write on a subject before understanding it fully.
> Thas is why I did this post.
>
> among others, a curious thing is that I perfectly remember that at the
> moment LBA was introduced, it was presented as a way to show CHS with
> fake numbers.
>
> I have yet to control this, but ATM most Bioses showed different CHS
> layout with LBA activated or not, but never shown direct sector
> numbers. It's only when Linux come than this direct adressing was used
> and never in partitionning tools.
>
> When fdisk advertise the 1024 cylinder barrier, I don't have found PC
> with this problem for more than 10 years now.
>
> My goal is to try making clear the bare minimum necessary to make
> actual partitionning on fixed and also external disks.
>
> as of LBA, this seems pretty cool:
> http://www.dewassoc.com/kbase/hard_drives/lba.htm
>
> thanks
> jdd


On Tue, Mar 17, 2009 at 3:25 AM, jdd <[hidden email]> wrote:
<snip>

>
> This discussion is very interesting, even if it go much more
> complicated I thought initially.
>
> I'm the "Partition Rescue" LDP HOWTO author and become recently the
> LDP coordinator. I was sometime asked to include my HOWTO in the
> Partition HOWTO and, as the Partition HOWTO author didn't want to
> continue with it I decided to take the HOWTO authorship.
>
> But the present HOWTO:
> http://tldp.org/HOWTO/Partition/index.html
> looks to me much too old, given all the new partitionning tools and
> disk usage we have now, so I began rewriting all this. It's *not* yet
> done. Drafts are here
> http://wiki.tldp.org/Partitions-and-mass-storage-HOWTO
>
> Some years ago, I made a course and for my students sake made use of
> loop mounted files as partition and file system testbed. I may use
> this also now for examples, but it's also easy to use usb keys, now so
> cheap user can (eventually) destroy one without real problem :-)
>
> But I don't want to write on a subject before understanding it fully.
> Thas is why I did this post.
>
> among others, a curious thing is that I perfectly remember that at the
> moment LBA was introduced, it was presented as a way to show CHS with
> fake numbers.
>
> I have yet to control this, but ATM most Bioses showed different CHS
> layout with LBA activated or not, but never shown direct sector
> numbers. It's only when Linux come than this direct adressing was used
> and never in partitionning tools.
>
> When fdisk advertise the 1024 cylinder barrier, I don't have found PC
> with this problem for more than 10 years now.
>
> My goal is to try making clear the bare minimum necessary to make
> actual partitionning on fixed and also external disks.
>
> as of LBA, this seems pretty cool:
> http://www.dewassoc.com/kbase/hard_drives/lba.htm
>
> thanks
> jdd
>
> --
> http://www.dodin.net
> http://valerie.dodin.org
> http://www.youtube.com/watch?v=t-eic8MSSfM
> http://www.facebook.com/profile.php?id=1412160445
>

JDD,

I think it is great you are working on that howto.  I consider myself
very knowledgeable about partitioning and I've learned a few things
about new concerns in just the last month or two.  Herbert's comment
about SDD's needing special partition alignment in particular was a
new concept for me.  And the 4k sector drive issue was new to me last
month.

Getting that info into a core HowTo like the LDP is great to see in
such a timely manner.

I suspect you've bitten off way more than you expected, but it is a
very worthwhile project.

Anyway I've scanned the draft and I have a couple hopefully
constructive comments:

1) Under software tools, I would like to see gpart listed and then a
link to later in the doc where you already explain how it is used.

2) As I assume you already realize, you need to add a section on
partition alignment.  As noted before, in the days when CHS
represented real disk geometry, partitions were aligned with the start
of a track.  Linux still defaults to that for the first partition for
no real reason as far as I know.  (Herbert implied bios'es and boot
code may still need that?)

Vista has changed the default to use 1 MiB offset into the disk for
the first partition and ignores CHS geometry altogether AIUI.  New
devices with atomic write units larger than 512 bytes have come to
market or soon will and need to be addressed.  They require special
alignment.  Apparently some new devices are trying to use CHS to
convey their alignment needs to the OS / partitioning tools.  Others
may not be.

2.1) Just yesterday I saw a post that some SSDs monitor the partition
table contents and realign themselves internally to make the alignment
work.  (That is just plain crazy to me, but it is what it is.)  Worthy
of a comment in the HowTo assuming it is true.

3) The HowTo talks in detail about the IBM PC partition table concept,
but does not even mention GPT or any other partitioning scheme (that I
noticed).  GPT is now the default partition table format used on Macs,
and Microsoft Vista supports it for data disks.  (ie. In general, not
supported for booting Vista from. Possibly one can make it work?)

The old IBM PC format maxes out at 2TB I believe, so GPT or something
else is coming.  I believe GPT is the recommended Linux Partition
table for drives (logical or physical) that exceed 2TB.  I think
*some* PC bios'es already support booting from GPT partitioned drives.
 Obviously the Mac line does.     (I don't think fdisk supports GPTs,
so another partitioning tool would have to be used.)

4) No reference to /proc/partitions.  (Is there an equivalent in /sys ?)

5) Is a section about future ideas appropriate?  In particular I
personally think all of the Linux partitioning tools need a conceptual
update to get away from CHS being treated as fundamental to the layout
of disk drives.  Even Microsoft has dropped that concept.

5.1) In the linux world, how do big conceptual changes like that come
about.  I'm familiar with LKML, and I assume there are individual
mailing lists for various partitioning tools, but if the Linux
community wanted to follow Microsoft's lead and start using 1 MiB as
the default starting point for new sectors, how would that decision be
reached and how would it get propagated out to the various tool
developers?

6) This thread http://markmail.org/message/tn5lfbf36dy6h2ox about 4K
sectors is worth at least you reading and possibly a good reference to
add to your doc.

Greg
--
Greg Freemyer
Head of EDD Tape Extraction and Processing team
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf

The Norcross Group
The Intersection of Evidence & Technology
http://www.norcrossgroup.com
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: fdisk calculations

jdd@dodin.org
In reply to this post by jdd@dodin.org
Greg Freemyer a écrit :

> I think it is great you are working on that howto.  I consider myself
> very knowledgeable about partitioning and I've learned a few things
> about new concerns in just the last month or two.

me too :-)

> 1) Under software tools, I would like to see gpart listed and then a
> link to later in the doc where you already explain how it is used.

of course, I ean to speak of every tool. That's not easy. For example,
most tools don't even try to partition loop mounted files when this is
a very interesting use (but may be not necessary with partitions),
when virtualization is so growing

>
> 2) As I assume you already realize, you need to add a section on
> partition alignment.

I even wonder if I shouldn't spread this HOWTO in several full HOWTOs
in place of only chapters.

In fact as I see now, there is room for a basic partition HOWTO,
assuming the tools do the smart things for the user (usually they do
the alignment work themselve) - most users wont even try to read
detailed doc as is needed (I just read the details barriers page
quoted by Wikipedia, very interesting :-) and an "advanced" HOWTO for
the experts

> Vista has changed the default to use 1 MiB offset...

the recommandation to always let Windows partition it's part of the
disk and only thereafter do the Linux part may be still valuable :-)

> 2.1) Just yesterday I saw a post that some SSDs monitor the partition
> table contents and realign themselves internally to make the alignment
> work.  (That is just plain crazy to me, but it is what it is.)  Worthy
> of a comment in the HowTo assuming it is true.

USB drives are more and more common - say externally connected drives,
to be more general (including SATA and IEEE1394). Booting external
drives is very challenging, so is the firmware of basic usb keys (I
still wonder how fidsk knows the full size of the usb key quoted in my
first post)

> table for drives (logical or physical) that exceed 2TB.

future system will have to be seen

> 4) No reference to /proc/partitions.  (Is there an equivalent in /sys ?)

some comments are on the definition document
>
> 5) Is a section about future ideas appropriate?

absolutely.

jdd
NB: what is the openSUSE partition tool? is it special or gpart
version? fidsk is rewritten now with parted libraries (seems to be
different than the util-linux used in openSUSE)




--
http://www.dodin.net
http://valerie.dodin.org
http://www.youtube.com/watch?v=t-eic8MSSfM
http://www.facebook.com/profile.php?id=1412160445

--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: fdisk calculations

Felix Miata
On 2009/03/17 22:56 (GMT+0100) jdd composed:

> the recommandation to always let Windows partition it's part of the
> disk and only thereafter do the Linux part may be still valuable :-)

I don't think it ever was a good idea. It's just an invitation to have
inconsistency among partitions when some other tool is used for the rest of
the disk. For years I've recommended picking one partitioning tool, and using
that only, to do 100% of all partitioning within a system. I've never run
into windoz or Linux or OS/2 having a partitioning problem as long as all
partitioning was done with a single tool.

http://fm.no-ip.com/partitioningindex.html
http://fm.no-ip.com/install-doz-after.html
--
"The plans of the diligent lead to profit as surely
as haste leads to poverty." Proverbs 21:5 NIV

 Team OS/2 ** Reg. Linux User #211409

Felix Miata  ***  http://fm.no-ip.com/
--
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

12