fsck on big UFS filesystems (>100TB)

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

fsck on big UFS filesystems (>100TB)

Гречихин Никита
Hi, everyone.
 
I have several servers with large UFS partitions (>100TB).
Physically it is RAID-6 on MegaRaid 9361.

If i try to run fsck on them, i get same error on all servers:
 
Code:

** Phase 5 - Check Cyl groups
fsck_ffs: inoinfo: inumber 18446744071562087424 out of range

inumber is exact same on all servers.
tested on clean and drity filesystems with same result.

And without succsessful run of fsck i can't remove dirty flag from filesystem
 
This is a fresh install of FreeBSD 12.1 and filesystem created in 12.1 from scratch with default options.
 
This is file storage for videoproduction. So, there is not too much files and they is relatively big.
 
Code:

# find . -type f -print | wc -l
  251485
 
# find . -type d -print | wc -l
   48351


Maybe i doing something wrong?
 
Code:

# uname -a
FreeBSD *** 12.1-RELEASE FreeBSD 12.1-RELEASE r354233 GENERIC  amd64

# gpart show mfid3
=> 40  250031243184  mfid3  GPT  (116T)
   40  250031243184      1  freebsd-ufs  (116T)

# df -i /mnt/data2
Filesystem      Size    Used   Avail Capacity iused ifree %iused  Mounted on
/dev/mfid3p1    116T     15T     91T    14%    300k  3.9G    0%   /mnt/data2

# tunefs -p /dev/mfid3p1
tunefs: POSIX.1e ACLs: (-a)                                disabled
tunefs: NFSv4 ACLs: (-N)                                   disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 enabled
tunefs: soft update journaling: (-j)                       enabled
tunefs: gjournal: (-J)                                     disabled
tunefs: trim: (-t)                                         disabled
tunefs: maximum blocks per file in a cylinder group: (-e)  4096
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: space to hold for metadata blocks: (-k)            8336
tunefs: optimization preference: (-o)                      time
tunefs: volume label: (-L)

# dumpfs /dev/mfid3p1 | head -30
magic   19540119 (UFS2) time    Sat Oct 10 04:46:19 2020
superblock location     65536   id      [ 5eab6d97 34710b41 ]
ncg     149965  size    31253905398     blocks  31006762468
bsize   32768   shift   15      mask    0xffff8000
fsize   4096    shift   12      mask    0xfffff000
frag    8       shift   3       fsbtodb 3
minfree 8%      optim   time    symlinklen 120
maxbsize 32768  maxbpg  4096    maxcontig 4     contigsumsize 4
nbfree  3369882144      ndir    48351   nifree  3915586242      nffree  36903
bpg     26051   fpg     208408  ipg     26112   unrefs  0
nindir  4096    inopb   128     maxfilesize     2252349704110079
sbsize  4096    cgsize  32768   csaddr  1672    cssize  2400256
sblkno  24      cblkno  32      iblkno  40      dblkno  1672
cgrotor 80650   fmod    0       ronly   0       clean   0
metaspace 8336  avgfpdir 64     avgfilesize 16384
flags   soft-updates+journal
check hashes    cylinder-groups
fsmnt   /mnt/data2
volname         swuid   0       providersize    31253905398

cs[].cs_(nbfree,ndir,nifree,nffree):
        (12392,3,26098,5) (11514,2,26104,4) (1034,4,26097,3) (5089,2,25977,7)
        (17633,1,26110,5) (992,7,26085,5) (14608,3,26098,5) (4047,3,26099,5)
        (17634,1,26110,5) (17636,1,26110,5) (13027,3,26099,0) (1089,3,26031,3)
        (13310,1,26106,1) (17632,1,26110,5) (8562,3,26098,5) (8185,1,26102,2)
        (14592,2,26107,3) (13173,2,26086,5) (13138,2,26020,2) (17687,1,26065,0)
        (13328,1,26016,7) (14754,1,26023,6) (13742,2,25995,1) (17825,1,26073,0)
        (6824,2,25958,4) (16919,2,26054,4) (1663,3,26098,5) (14690,3,26070,4)
        (2495,3,26099,4) (17636,3,26108,1) (10462,3,26099,3) (17631,1,26110,5)

 
 
--
WBR,
Nikita Grechikhin
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: fsck on big UFS filesystems (>100TB)

Stefan Esser-3
Am 14.10.20 um 18:19 schrieb Nikita Grechikhin:

> Hi, everyone.
>  
> I have several servers with large UFS partitions (>100TB).
> Physically it is RAID-6 on MegaRaid 9361.
>
> If i try to run fsck on them, i get same error on all servers:
>  
> Code:
>
> ** Phase 5 - Check Cyl groups
> fsck_ffs: inoinfo: inumber 18446744071562087424 out of range
This number is near minus 2^31 - without having looked at the
code this smells like an integer overflow.
> inumber is exact same on all servers.
> tested on clean and drity filesystems with same result.
>
> And without succsessful run of fsck i can't remove dirty flag from filesystem
>  
> This is a fresh install of FreeBSD 12.1 and filesystem created in 12.1 from scratch with default options.

The default options assume an average file size of just a few KB,
and I severely doubt you are going to fill 100 TB with small files.

> This is file storage for videoproduction. So, there is not too much files and they is relatively big.
>  
> Code:
>
> # find . -type f -print | wc -l
>    251485
>  
> # find . -type d -print | wc -l
>     48351

What fraction of the storage is filled?

You may need to recreate the file system with an appropriate
value if "-i", the number of bytes per inode.

If your files are generally in the order of 1 GB, then you
should try -i 100000000 (i.e. 100 mio to allow for up to
1 million files to have some slack).

> Maybe i doing something wrong?
>  
> Code:
>
> # uname -a
> FreeBSD *** 12.1-RELEASE FreeBSD 12.1-RELEASE r354233 GENERIC  amd64
>
> # gpart show mfid3
> => 40  250031243184  mfid3  GPT  (116T)
>     40  250031243184      1  freebsd-ufs  (116T)
>
> # df -i /mnt/data2
> Filesystem      Size    Used   Avail Capacity iused ifree %iused  Mounted on
> /dev/mfid3p1    116T     15T     91T    14%    300k  3.9G    0%   /mnt/data2
Yes, way too many inodes (4G) - you are wasting space and run
into the fsck issue due to them ...

Regards, STefan

OpenPGP_signature (505 bytes) Download Attachment